Getting Started

From Obsidian Scheduler
Jump to: navigation, search

Important: This Getting Started is for Obsidian 4.0 and newer versions. See the Previous Getting Started for prior versions.

This guide covers your initial installation and setup of Obsidian Scheduler. It also covers the startup and deployment options available to you, and various tweaks you can make.

You should have already downloaded Obsidian Scheduler. If you have not done so, see Downloads.

If you are already running Obsidian, see our upgrade instructions.

If you're looking to start writing your own Obsidian jobs, see Implementing Jobs.

Quick Start

If you wish to evaluate or quickly check out Obsidian, here are the steps you need to follow:

  1. Ensure you have a JDK (1.7 or up) installed and that the JAVA_HOME environment variable is set to your JDK installation directory.
  2. Create database for use by Obsidian (MySQL/MariaDB 5.5, Oracle 10g, PostgreSQL 9 or MS SQL Server 2008 or up). For quick evaluation you can also use H2 in-memory or filesystem database.
  3. Unzip the Obsidian package zip file.
  4. Run the installer UI by using the executable jar java -jar Obsidian-Install-n-n-n.jar or in interactive console mode java -jar Obsidian-Install-n-n-n.jar -console.
  5. Apart from your database configuration values (JDBC URL, user, password), most default values should be fine.
  6. Deploy the admin/scheduler war to your chosen servlet container (Tomcat, Jetty or similar).
    1. The Jetty configuration defaults its http listener port to 8080.
  7. Log in to the admin web application with the default user admin and password changeme.

Really Quick Start With H2 and Jetty

If you wish to quickly try Obsidian without setting up a database or web server, you can take advantage of our really quick config using an H2 database and Jetty web server support.

  1. Ensure you have a JDK (1.7 or up) installed and that the JAVA_HOME environment variable is set to your JDK installation directory.
  2. Unzip the Obsidian package zip file.
  3. Run the installer by using the executable jar and the autoinstall xml. java -jar Obsidian-Install-n-n-n.jar h2-jetty-quick-start.xml
  4. Start Obsidian using the Jetty starter scripts: ./webObsidian.sh start scheduler or webObsidian.bat start scheduler
  5. Go to http://localhost:8080 and check out your fully functional Obsidian web app and scheduler! You can log in to the admin web application with the default user admin and password changeme.
  6. Stop Obsidian using the Jetty starter scripts: ./webObsidian.sh stop or webObsidian.bat stop

Note: If you restart the quick start instance using H2 and see "lock wait timeout" or similar errors in the log screen, you may have to delete your H2 lock file. By default, the file name is obsidian.lock.db and is located in user/home directory.

Note: If you see a java.net.BindException: Permission denied error on startup, you will need to use an alternate port. See Troubleshooting for details on how to change this.

Supported Platforms

  • Obsidian is OS-independent.
  • Obsidian runs on Java 1.7 or above.
  • Obsidian is fully tested on MySQL 5.1 & 5.5, MariaDB 5.5, Oracle 10g & 11.x, PostgreSQL 9, MS SQL Server 2008 & 2014 and H2. Larger major versions are likely to work but are not supported.
  • Obsidian's administration web application is tested on Tomcat 7 & 8 and Jetty 9.x, but is likely to work in any compliant servlet container version 2.4 such as WebSphere. It works on all modern browsers and is tested on recent versions of Chrome and Firefox, plus IE 10.

Note: Clustered scheduler instances must have access to contact license servers over the Internet, or access to an internal proxy license server that you may set up. See Licenses & Nodes for more information.

Hardware Requirements

We recommend the following minimum hardware for typical installations:

  • 1 GHz processor
  • 2 GB RAM
  • 1 GB hard disk space (excluding accumulated logs)

Note that actual requirements may vary depending on job volume, clustering and other factors.

Choose Your Installation Type

If you skipped the Quick Start section or wish more control over your installation, the following sections will detail your choices and steps to configure and deploy Obsidian.

Obsidian consists of two main processes:

  • Scheduler
  • Admin Web Application


These can be run together or in separate processes in the following configurations:

  1. Standalone Scheduler
  2. Embedded Scheduler (within your application)
  3. Standalone Admin Web Application
  4. Combined Scheduler and Admin Web Application


Choosing which option works best for you depends on your specific needs. The setups which are suitable for most cases would be:

  • Using an embedded scheduler (option 2) along with a standalone admin webapp (option 3). This approach lets you embed Obsidian without having to alter the Obsidian web application to include your job code. See Classpath Notes.
  • Using a combined scheduler and admin web app (option 4). This will require adding your job code to the Obsidian WAR file in the form of class or jar files.


You may wish to use multiple configurations together. For example, you may combine standalone schedulers along with a combined scheduler and web application.

Note: A key server proxy package also exists and you may choose to deploy it within a servlet container like Jetty or Tomcat. See Key Server Proxy for details on key server proxies.

Initial Setup

This section covers the setup required after you've selected your deployment setup.

Database

Obsidian relies on a database which should be configured before attempting to deploy your scheduler processes.

Currently Obsidian supports the following databases:

  • MySQL 5.5
  • MariaDB 5.5
  • Oracle 10g & 11.x
  • PostgreSQL 9
  • MS SQL Server 2008, 2012 and 2014.
  • H2 (recommended for evaluations only)

To configure which database Obsidian will use, specify the connection parameters when prompted to do so in the installer.

To review a detailed breakdown of the resulting configuration, see Advanced Configuration. To make use of our external property file override support, use the System property carfey.properties.file. Between the default properties and the override, all expected properties must be specified. Any properties found in both files will use the override file's values. Usage: -Dcarfey.properties.file=/home/obsidian/obsidian.properties.

Note that the database must exist before deployment, but Obsidian will automatically perform initial setup including table creation. When you first deploy your scheduler web app, either standalone admin or embedded scheduler, it will perform all necessarily initialization.

Note: Obsidian needs to create the tables in the target database. If the schema is shared with your application’s tables, please ensure there are no name conflicts. If there are conflicts, separate schemas/databases can be used, or a table prefix can be specified as shown in Advanced Configuration. See the Obsidian Tables that are created upon first deployment.

Alternate Oracle Schemas

After allowing Obsidian to create all the required tables and initial data by deploying it a single time, Obsidian can be run with an alternate user by specifying the target schema on Oracle databases. See Advanced Configuration for the property to set in the installation properties file.

If you use a separate user from the schema owner, you must grant the user certain privileges for Obsidian to function correctly.

MS SQL Server Snapshot Isolation

For maximum compatibility and to avoid deadlocks, MS SQL Server should be configured to use read committed snapshot isolation.

This can be enabled on your database by running the following commands:

ALTER DATABASE MyDatabase SET ALLOW_SNAPSHOT_ISOLATION ON
ALTER DATABASE MyDatabase SET READ_COMMITTED_SNAPSHOT ON


When you start Obsidian, it will automatically detect and create missing tables in your target database. However, if you wish to run Obsidian with a database user without create privileges, or simply wish to prepare your database structure ahead of time, you can use the database scripts provided with your download zip file under the "db_scripts" directory.

First create your database, and locate the "complete.sql" file under the directory for the database you are using under the "db_scripts" directory. Statements within this file contain the token "$px$" to enable table prefixes. Before you run the SQL file, replace all instances of the token "$px$" (without quotes) with a table prefix 6 characters or shorter (letters and underscores supported), or with blank.

Then, in your SQL client of choice, execute the script and all the required Obsidian tables will be created.

Authentication

The Obsidian admin web application supports both native users and LDAP-enabled authentication or you can implement your own authenticator.

By default, it is configured to use native login, and a default “admin” user is created when the scheduler is first deployed. Users can then be created and managed from within the admin management console.

To use LDAP-based authentication, choose LDAP authentication and provide the requested configuration values during the installation.

To review a detailed breakdown of the resulting configuration, see Advanced Configuration.

Deployment

You are now ready to deploy your scheduler and/or admin web application. You may first wish to review your high-level deployment options.

Before deploying Obsidian, you'll have to obtain the installation package which includes various deployment artifacts at Downloads.

Classpath Notes

To run a scheduler with your custom code and jobs, you need to ensure the scheduler process classpath includes your jars. If a compiled job is updated, you will have to restart your application or the servlet container after deploying updated jars. For details on setting your classpath, follow these guidelines.

To implement jobs in Java, you will need to reference Obsidian base classes in your Java project. In addition, to run jobs, your built code and any 3rd party libraries it requires must be included in the Obsidian servlet container (e.g. Tomcat) under /WEB-INF/lib. This could be a plain installation of Obsidian, or a web app which contains both your application and Obsidian.

If you are running a standalone web application that does not have a scheduler running, you do not have to update its classpath with your compiled jobs, unless you are running a version older than 2.6. To be able to configure jobs in your standalone web application, your Obsidian instances which run jobs will need to have been started at least once after your latest classpath changes, and after configuring classpath scanning. This is because scheduler instances store job metadata in the Obsidian database so the admin web application can properly validate jobs.

If you are embedding Obsidian into your application, you will need to reference all the base Obsidian JAR files for scheduling to function.

Setting Host Names

Obsidian instances will automatically assign themselves host names if no host name is explicitly set, but you may wish to give them explicit names to make scheduling and monitoring simpler.

If you wish to assign explicit names, simply set the Java system property schedulerDesignation to the host name of your choice. For example, if starting an instance using the standalone scheduler using java directly, simply add the value -DschedulerDesignation=myHostName to the end of the command.

If you have a need to set multiple different host names within a single JVM, use a property configuration as detailed in (Advanced Configuration).

Deployment Models

Standalone Scheduler

To deploy and run the standalone scheduler, you may use the provided standalone starter scripts ./standaloneObsidian.sh start or standaloneObsidian.bat start - these use 10451 as their Obsidian listener port. To stop Obsidian, pass the stop argument: ./standaloneObsidian.sh stop or standaloneObsidian.bat stop.

You can always use your preferred means to invoke the main starter method as follows:

java com.carfey.ops.job.SchedulerStarter start <listenerPort>

Listener port is a port number used by Obsidian to subsequently issue a shut down:

java com.carfey.ops.job.SchedulerStarter stop <listenerPort>

Startup and Shutdown example:

java com.carfey.ops.job.SchedulerStarter start 10451
java com.carfey.ops.job.SchedulerStarter stop 10451

Embedded Scheduler

To embed Obsidian within your application, you will need to include the following dependencies on the classpath:

  • com.carfey.properties file. This can be found in obsidian.war under WEB-INF/classes or obsidian-props.jar from your standalone directory.
  • All required dependent libraries. Unused JDBC JAR files may be left out.

Then, the embedded scheduler can be run in any existing Java application by using the included scheduler starter to initialize and shut down the scheduler processes:

// will start scheduler on first call to get()

com.carfey.ops.job.SchedulerStarter starter = com.carfey.ops.job.SchedulerStarter.get(com.carfey.ops.job.SchedulerStarter.SchedulerMode.EMBEDDED);

// ...

// later, we can gracefully shut down the scheduler 
starter.shutDown();

This will run Obsidian's scheduler process, but will not include any of the web application or the REST API. To embed all of this functionality in your web application, see the next section.

Embedding the Obsidian Web Application in Another Web Application

If you want to embed the full Obsidian web application in an existing web application, you will need to extract the necessary sections from the WEB-INF/web.xml file in the obsidian.war file and merge them into your application's web.xml file. This includes all "listener", "servlet", "servlet-mapping", "jsp-config", "filter", "filter-mapping" elements, and optionally "welcome-file-list" and "error-page" elements. Ensure that all servlet paths are maintained. If you are using a different servlet specification version, you may need to update the mappings to the appropriate format.

Note the use of our com.carfey.ops.servlet.StartupShutdownListener which takes care of starting up the scheduler instance and using the web container's default shutdown mechanism to ensure graceful shutdown. This listener will start Obsidian's scheduler process and initialize the Obsidian web application.

If you wish to embed Obsidian's web application without the scheduler running, include the following parameter element in your web.xml:

<context-param>
   <param-name>schedulerEnabled</param-name>
   <param-value>false</param-value>
</context-param>

Combined Scheduler and Admin Web Application

To deploy and run the combined scheduler and admin web application, simply deploy the obsidian.war file to your servlet container (e.g. Jetty, Tomcat) after configuring it appropriately using the UI installer. You may also need to package your custom jars in with the war so the scheduler can run your custom jobs.

Security Note: By default, the admin web application allows non-secure connections. If you wish to force secure connections through HTTPS, you can edit the web.xml in your war and uncomment the <security-constraint> element in the file. This will force all requests to redirect to an encrypted connection. For details on setting up SSL on your servlet container, refer to its documentation.

Note: If using Jetty to enable running Obsidian without setting up Tomcat or another servlet container, you can use the Jetty starter scripts. ./webObsidian.sh start scheduler or webObsidian.bat start scheduler. To stop, do ./webObsidian.sh stop or webObsidian.bat stop.

Standalone Admin Web Application

As with the combined scheduler and admin web application, to deploy the standalone admin web application, simply deploy the standalone admin war to your servlet container after configuring it appropriately using the UI Installer or by editing the com.carfey.properties file within the WAR directly found at WEB-INF/classes/com.carfey.properties.

Note: If using Jetty to enable running Obsidian without setting up Tomcat or another servlet container, you can use the Jetty starter scripts. ./webObsidian.sh start adminOnly or webObsidian.bat start adminOnly. To stop, do ./webObsidian.sh stop or webObsidian.bat stop

Disabling Job Execution & Scheduling in the Web Application

If you have web application package already built (i.e. a WAR file), you can easily tweak it to enable or disable the scheduler process.

Simply add the following to your WEB-INF/web.xml file after any listener elements:

<context-param>
    <param-name>schedulerEnabled</param-name>
    <param-value>false</param-value>
</context-param>

Disabling Automatic Database Updates

Obsidian automatically applies schema updates and data upgrades to its database on startup. In some cases, it may be desirable to disable this once the database has been fully initialized and upgrades the new versions are not going to be deployed. For example, you may wish to run Obsidian with a user who does not have privileges to modify the database schema.

To do so, you can either set a Java system property, or if you are using the web application WAR, you can add a setting to your WEB-INF/web.xml.

As a system property:

-DstartupRunnerClass=com.carfey.ops.run.NullRunner

In WEB-INF/web.xml, after any listener elements:

<context-param>
    <param-name>startupRunnerClass</param-name>
    <param-value>com.carfey.ops.run.NullRunner</param-value>
</context-param>

You’re Good to Go!

You are now ready to run the admin web app! As the default admin user if using native authentication or a designated admin user in LDAP, login to the admin web app, navigate to the scheduler settings screen, and review/adjust system settings as needed. Defaults are provided for all settings.

If you are an admin user, you can log into the admin web application and customize your installation settings using the scheduler settings screen. This has advanced options that allow you to customize scheduler and administration settings.

To get started writing jobs that you can run in Obsidian, see Implementing Jobs.

See the User Guide if you have questions or wish to explore what Obsidian offers.

Default User: Each Obsidian installation using native authentication (the default) starts with a single default user named admin with password changeme. You should change this after your first login. See User Management for how to change a user's password.