On some projects, you find you need to execute certain jobs and tasks at an exactly specified time or at regular time intervals. In this article we will see how Java developers can implement such a requirement using the standard Java Timer API, and then we will focus on Quartz, an open source library for those who need some extra features in their scheduling system.
Let's, for a start, go through few common use cases that will help you recognize situations when you could need this kind of behavior. Then we will see how to find the best solution according to your functional request.
Almost every business application has some reports and statistics that are needed by its users. You can hardly imagine such a system without these reports, because the common purpose for someone to invest in such a system is the ability to collect a large amount of data and see it in a manner that can help with further business planning. A problem that can exist in creating these reports is the large amount of data that needs to be processed, which would generally put a heavy load on the database system. This heavy load decreases overall application performance and affects users that only use the system for data collecting, making it practically useless while it's generating the reports. Also, if we think about being user-friendly, a report that takes ten minutes to generate is not an example of a good response time.
We will now focus on the type of reports that can be cached, meaning those that are not needed in real time. The good news is that most reports fall into this category -- statistics on some product sales in last June, or company income in January. For these cacheable reports, a simple solution is possible: schedule report creation for times when the system is idle, or at least when the load on the database system is minimal. Let's say that you are creating an application for a local book reseller that has many offices (all in the same time zone) and that you need to generate a (possibly large) report for weekly income. You could schedule this task of generating necessary data for a time when the system isn't being used, such as every Sunday at midnight, and cache it in the database. This way, no sale operators will notice any performance problem in your application and company management will have all necessary data in a moment.
A second example that I will mention here is sending all kinds of notifications to application users, such as account expirations. This could be done using date fields in the user's data row, and creating a thread to check users on that condition, but using a scheduler in this case is surely a more elegant solution, and better from the aspect of overall system architecture (and that's important, right?). In a complex system you would have a lot of these notifications, and could find that you need a scheduler-like behavior in many other cases, so building a specific solution for every case makes a system harder to change and maintain. Instead, you should use one API to handle all of your application scheduling. That is the topic of the rest of this article.
In-House Solution
To implement a basic scheduler in your Java application, you don't need any external library. As of J2SE 1.3, Java contains thejava.util.Timer and java.util.TimerTask classes that can be used for this purpose. Let's first make a little example that will help us describe all of the possibilities of this API.package net.nighttale.scheduling;
import java.util.Calendar;
import java.util.Date;
import java.util.Timer;
import java.util.TimerTask;
public class ReportGenerator extends TimerTask {
public void run() {
System.out.println("Generating report");
//TODO generate report
}
}
class MainApplication {
public static void main(String[] args) {
Timer timer new Timer();
Calendar date = Calendar.getInstance();
date.set(
Calendar.DAY_OF_WEEK,
Calendar.SUNDAY
);
date.set(Calendar.HOUR, 0);
date.set(Calendar.MINUTE, 0);
date.set(Calendar.SECOND, 0);
date.set(Calendar.MILLISECOND, 0);
// Schedule to run every Sunday in midnight
timer.schedule(
new ReportGenerator(),
date.getTime(),
1000 * 60 * 60 * 24 * 7
);
}
}The code above implements the example from the introduction, scheduling report generation for a time when the system is idle (in this case, Sunday at midnight).
First, we should implement a "worker" class that will actually do the scheduled job. In our example this is
ReportGenerator. This class must extend from java.util.TimerTask, which implements java.lang.Runnable. All that you need to do is to override run() method with the report generation code.Then, we schedule this object's execution using one of the
Timer's scheduling methods. In this case, we use the schedule() method that accepts the date of the first execution and the period of subsequent executions in milliseconds (since we repeat this report generation every week).As we use the scheduling features, we must be aware of the real-time guarantees that the scheduling API provides. Unfortunately, because of the nature of Java and its implementations on various platforms, thread scheduling implementations in various JVMs are inconsistent. Thus, the
Timer cannot guarantee that our Task will be executed at exactly the specified moment. Our Tasks are implemented as Runnable objects and are put to sleep (with wait) for some time. Timer then wakes them up at a specified moment, but the exact moment of execution depends on the JVM's scheduling policy and how many threads are currently waiting for the processor. There are two common cases that can cause our tasks to be executed with a delay. First, a large number of threads might be waiting to be executed; second, a delay could be caused by garbage collection activity. All of these influences could be minimized using different techniques (such as running a scheduler within a different JVM or tuning some options for the garbage collector) but that is beyond the topic of this article.In this new light, we can explain the existence of two different scheduling method groups in the
Timer class: scheduling with fixed delay (schedule()) and scheduling with fixed rate (scheduleAtFixedRate()). When you are using methods from the first group, every delay in the task execution will be propagated to the subsequent task executions. In the latter case, all subsequent executions are scheduled according the time of the initial task execution, hopefully minimizing this delay. Which methods you use depends on which parameter is more important in your system.One more thing is very important to say here: every
Timer object starts a thread in the background. This behavior is not desirable in a managed environment, such as a J2EE application server, because these threads are not in the scope of the container.Beyond the Ordinary
So far, we saw how we can schedule tasks in an application, and for simple requirements that is quite enough. But for more advanced users and complex requirements, a lot more features are needed to support fully useful scheduling. In such cases, there are two common solutions that one could choose. The first is to build your own scheduler with the desired functionality; the second is to locate a project that can fulfill your specific needs. The second solution is more appropriate in most cases, because you can save time and resources and won't have to duplicate someone else's effort.This brings us to Quartz, an open source job-scheduling system with significant advantages over the simple Timer API.
The first advantage is persistence. If your jobs are "static," as in our introductory example, then maybe you don't need to persist jobs. But we often find tasks need to be "dynamically" triggered when a certain condition is met, and you have to be sure that those tasks won't be lost between system restarts (or crashes). Quartz offers both non-persistent and persistent jobs, in which state is saved in a database, so you can be sure that those jobs won't be lost. Persistent jobs introduce additional performance overhead in the system, so you have to use them pragmatically.
The Timer API also lacks methods to simplify setting the desired execution time. As we saw in the example above, the most you can do is to set a start date and a repeat interval. Surely, everyone who has used the Unix
cron tool would like to see similar configuration possibilities within their scheduler. Quartz defines org.quartz.CronTrigger, which lets you set a desired firing date in a flexible manner. Developers often need one more feature: managing and organizing jobs and tasks by their names. In Quartz, you can get jobs by their names or presence in a group, which can be a real help in environments with a large number of scheduled jobs and triggers.
Now, let's implement our report generation example using Quartz and explain basic features of the library.
package net.nighttale.scheduling;
import org.quartz.*;
public class QuartzReport implements Job {
public void execute(JobExecutionContext cntxt)
throws JobExecutionException {
System.out.println(
"Generating report - " +
cntxt.getJobDetail().getJobDataMap().get("type")
);
//TODO Generate report
}
public static void main(String[] args) {
try {
SchedulerFactory schedFact
new org.quartz.impl.StdSchedulerFactory();
Scheduler sched schedFact.getScheduler();
sched.start();
JobDetail jobDetail
new JobDetail(
"Income Report",
"Report Generation",
QuartzReport.class
);
jobDetail.getJobDataMap().put(
"type",
"FULL"
);
CronTrigger trigger new CronTrigger(
"Income Report",
"Report Generation"
);
trigger.setCronExpression(
"0 0 12 ? * SUN"
);
sched.scheduleJob(jobDetail, trigger);
} catch (Exception e) {
e.printStackTrace();
}
}
}Jobs and Triggers. A Job is the abstraction of the work that should be done, and Trigger represents time when the action should occur.Job is an interface, so all we have to do is to make our class implement the org.quartz.Job interface (or org.quartz.StatefulJob, as we will see in a moment) and override the execute() method. In the example, we saw how one can pass parameters to the Job through the jobDataMap attribute, which is a modified implementation of java.util.Map. Deciding whether you are going to implement stateful or non-stateful jobs is a matter of deciding whether you want to change these parameters during execution. If you implement Job, all of the parameters are saved at the moment the job is scheduled for the first time, and all changes made later are discarded. If you change the StatefulJob's parameters in the execute() method, this new value will be passed when the job triggers another time. One important implication to consider: StatefulJobs can't be executed concurrently, because the parameters might change during the execution.There are two basic kinds of
Triggers: SimpleTrigger and CronTrigger. SimpleTrigger provides basically the same functionality you get from the Timer API. It should be used if the Job should be triggered once, followed possibly by repeats at a specific interval. You can specify start date, end date, repeat count, and repeat interval for this kind of trigger. In the example above, we used
CronTrigger because of its flexibility to schedule jobs on a more realistic basis. CronTriggers allow us to express schedules such as "every weekday at 7:00 p.m." or "every five minutes on Saturday and Sunday." We will not go further into explaining cron expressions in this article, so you are advised to find all of the details about its possibilities in the class' Javadoc. In order to run the above example, you'll need a file named quartz.properties in the classpath, with basic Quartz configuration. If you want to use a different name for the file, you should pass it as a parameter to the
StdSchedulerFactory constructor. Here is an excerpt of the file with minimal properties:#
# Configure Main Scheduler Properties
#
org.quartz.scheduler.instanceName = TestScheduler
org.quartz.scheduler.instanceId = one
#
# Configure ThreadPool
#
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 5
org.quartz.threadPool.threadPriority = 4
#
# Configure JobStore
#
org.quartz.jobStore.misfireThreshold = 5000
org.quartz.jobStore.class = org.quartz.simpl.RAMJobStoreJobs. There is one thread pool implementation in Quartz itself, but you are not limited on its usage. Now we come to the
JobStores. JobStores keep all the data about Jobs and Triggers. So, this is where we need to decide if we are going to keep our Jobs persistent or not. In the example, we used org.quartz.simpl.RAMJobStore, which means that all of the data will be kept in memory and thus will not be persistent. As a result, if the application crashes, all the data about scheduled Jobs will be lost. In some situations, this is the desired behavior, but when you want to make the data persistent, you should configure your application to use org.quartz.simpl.JDBCJobStoreTX (or org.quartz.simpl.JDBCJobStoreCMP). JDBCJobStoreTX requires some more configuration parameters that will be explained by example.org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
org.quartz.jobStore.dataSource = myDS
org.quartz.jobStore.tablePrefix = QRTZ_
#
# Configure Datasources
#
org.quartz.dataSource.myDS.driver = org.postgresql.Driver
org.quartz.dataSource.myDS.URL = jdbc:postgresql:dev
org.quartz.dataSource.myDS.user = dejanb
org.quartz.dataSource.myDS.password =
org.quartz.dataSource.myDS.maxConnections = 5 After that, we should declare the delegate class that adapts standard SQL queries to the specific RDBMS SQL dialect. In our example, we have chosen PostgreSQL as the database server so the appropriate
org.quartz.impl.jdbcjobstore.PostgreSQLDelegate class is submitted. For information what delegate to use with your specific server, you should consult the Quartz documentation. The
tablePrefix parameter defines what prefix is used for Quartz tables in the database (the default is QRTZ_). This way, you can distinguish these tables from the rest of the database. For every JDBC store, we need to define datasource to be used. This is part of the common JDBC configuration and will not be further explained here.
The beauty of Quartz is that after these configuration changes are made, our report-generation example would persist the data in the database without changing a single line of code.
Advanced Quartz
So far we have described basic Quartz features that can be a good foundation to start using it in your projects. Besides that, this library has great architecture that can fully leverage your effort.Besides the basics provided by Quartz, the library has a great architecture that will help solve problems in your application. One of its key features is listeners: objects that are called when some event in the system occurs. There are three types of listeners:
JobListeners, TriggerListeners, and SchedulerListeners. Listeners can be particularly useful when you want a notification if something goes wrong in the system. For example, if an error occurs during report generation, an elegant way to notify the development team about it is to make a
JobListener that will send an email or SMS. A
JobListener can provide more interesting functionality. Imagine a Job that has to deal with a task that is highly dependent on some system resource availability (such as a network that is not stable enough). In this case, you can make a listener that will re-trigger that job if the resource is not available when the job is executed. Quartz can also deal with situations when some
Trigger has misfired, or didn't fire because the scheduler was down. You can set a misfire instruction by using the setMisfireInstruction() method of the Trigger class. It takes the misfire instruction type for a parameter, which can have one of the following values:Trigger.INSTRUCTION_NOOP: does nothing.Trigger.INSTRUCTION_RE_EXECUTE_JOB: executes the job immediately.Trigger.INSTRUCTION_DELETE_TRIGGER: deletes the misfired trigger.Trigger.INSTRUCTION_SET_TRIGGER_COMPLETE: declares the trigger completed.Trigger.INSTRUCTION_SET_ALL_JOB_TRIGGERS_COMPLETE: declares all triggers for that job completed.Trigger.MISFIRE_INSTRUCTION_SMART_POLICY: chooses the best fit misfire instruction for a particularTriggerimplementation.
Trigger implementations (such as CronTrigger) can define new types of misfire instructions that can be useful. You should check out the Javadocs for these classes for more information on this topic. Using the TriggerListener, you can gain more control on actions that should be used if a misfire occurs. Also, you can use it to react to trigger events, such as a trigger's firing and completion. SchedulerListener deals with global system events, such as scheduler shutdown or the addition or removal of jobs and triggers. Here, we will just demonstrate a simple
JobListener for our report generation example. First we have to write a class to implement the JobListener interface.package net.nighttale.scheduling;
import org.quartz.*;
public class MyJobFailedListener implements JobListener {
public String getName() {
return "FAILED JOB";
}
public void jobToBeExecuted
(JobExecutionContext arg0) {
}
public void jobWasExecuted(
JobExecutionContext context,
JobExecutionException exception) {
if (exception != null) {
System.out.println(
"Report generation error"
);
// TODO notify development team
}
}
}sched.addGlobalJobListener(new MyJobFailedListener());Scheduler's addJobListeners() method to register the listener with the scheduler. Then add the registered listener to the job's list of listeners with JobDetail's addJobListener() method, with the listener name as a parameter (the value returned from getName() method of the listener).sched.addJobListener(new MyJobFailedListener());
jobDetail.addJobListener("FAILED JOB");throw new JobExecutionException();execute() method of the report generation job. After the job has been executed, the jobWasExecuted() method of our listener is executed, and the thrown exception is passed as the argument. Because the exception is not null in our case, you should expect to see the message "Report generation error" on the screen. One final note about listeners is that one should be careful about number of listeners that is used in the system, because it could down the performance of the application.
There is one more way you can extend Quartz's features, and that is through plug-ins. Plug-ins can do practically any work you need; all you have to is to write the class implementing the
org.quartz.spi.SchedulerPlugin interface. This interface defines two methods that need to be implemented -- one for initialization (which takes a Scheduler object as a parameter) and one for shutdown. Everything else is up to you. In order to make SchedulerFactory use a certain plug-in, all you have to do is to add a line in the properties file (quartz.properties) with the plug-in class and a few optional configuration parameters (which depend on the particular plug-in). There are a few plug-ins already in Quartz itself. One is the shutdownHook, which can be used to cleanly shut down the scheduler in case the JVM terminates. To use this plug-in, just add the following lines in the configuration file:org.quartz.plugin.shutdownHook.class =
org.quartz.plugins.management.ShutdownHookPlugin
org.quartz.plugin.shutdownHook.cleanShutdown = trueAdaptable in Every Environment
All of the above applies to using Quartz in a standalone application. Now, we will see how we can use its interface in some of the most common environments for Java developer.RMI
With a distributed application using RMI, Quartz is simple, as we've seen above. The differences are in the configuration.There are two necessary steps: first we have to set Quartz as an RMI server that will handle our requests, and then we just use it in the standard manner.
The source code of this example is practically the same as in our first example, but here we will divide it in two parts: scheduler initialization and scheduler usage.
package net.nighttale.scheduling.rmi;
import org.quartz.*;
public class QuartzServer {
public static void main(String[] args) {
if(System.getSecurityManager() != null) {
System.setSecurityManager(
new java.rmi.RMISecurityManager()
);
}
try {
SchedulerFactory schedFact =
new org.quartz.impl.StdSchedulerFactory();
Scheduler sched = schedFact.getScheduler();
sched.start();
} catch (SchedulerException se) {
se.printStackTrace();
}
}
}#
# Configure Main Scheduler Properties
#
org.quartz.scheduler.instanceName = Sched1org.quartz.scheduler.rmi.export = true
org.quartz.scheduler.rmi.registryHost = localhost
org.quartz.scheduler.rmi.registryPort = 1099
org.quartz.scheduler.rmi.createRegistry = true
#
# Configure ThreadPool
#
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 5
org.quartz.threadPool.threadPriority = 4
#
# Configure JobStore
#
org.quartz.jobStore.misfireThreshold = 5000
org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore We have do a few more things to successfully deploy our server, all of which are typical tasks for exporting objects through RMI. First, we need to start
rmiregistry on the server (if it is not already up). This is done by calling: rmiregistry & start rmiregistryNext, start the
QuartzServer class with the following options (all on one line):java -Djava.rmi.server.codebase
file:/home/dejanb/quartz/lib/quartz.jar
-Djava.security.policyrmi.policy
-Dorg.quartz.propertiesquartzServer.properties
net.nighttale.scheduling.rmi.QuartzServerrmic call to create the necessary RMI classes, so to point clients to the codebase with these classes, you have to start it with the -Djava.rmi.server.codebase parameter set to file: plus the full path to the location of quartz.jar (of course, this could also be a URL to the library).An important issue in a distributed system is security; thus, RMI enforces that you use security policy. In this example, we used the basic policy file (rmi.policy) that grants all privileges.
grant {
permission java.security.AllPermission;
};OK, now the
Scheduler is ready to accept your Jobs via RMI. Let's write the client that will do that.package net.nighttale.scheduling.rmi;
import org.quartz.*;
import net.nighttale.scheduling.*;
public class QuartzClient {
public static void main(String[] args) {
try {
SchedulerFactory schedFact =
new org.quartz.impl.StdSchedulerFactory();
Scheduler sched = schedFact.getScheduler();
JobDetail jobDetail = new JobDetail(
"Income Report",
"Report Generation",
QuartzReport.class
);
CronTrigger trigger = new CronTrigger(
"Income Report",
"Report Generation"
);
trigger.setCronExpression(
"0 0 12 ? * SUN"
);
sched.scheduleJob(jobDetail, trigger);
} catch (Exception e) {
e.printStackTrace();
}
}
} # Configure Main Scheduler Properties
org.quartz.scheduler.instanceName = Sched1
org.quartz.scheduler.rmi.proxy = true
org.quartz.scheduler.rmi.registryHost = localhost
org.quartz.scheduler.rmi.registryPort = 1099No other settings are necessary, because all the work is done on the server side. In fact, if other settings are present, they will be ignored. The important thing is that the names of the schedulers must match in the client and server configurations (
Sched1 in this case). And that's all there is, for a start. Just run the client with redirected properties file (again, all on one line): java -Dorg.quartz.properties
quartzClient.properties
net.nighttale.scheduling.rmi.QuartzClientand you should expect the same behavior in the server console as in the first example.
Web and Enterprise
If you are developing a web or enterprise solution, one question that may come up is the right place to start the scheduler. For that, Quartz providesorg.quartz.ee.servlet.QuartzInitializerServlet. All we need to do is to make the following configuration in the web.xml file:<servlet> <servlet-name>QuartzInitializer</servlet-name> <display-name>Quartz Initializer Servlet</display-name> <servlet-class>org.quartz.ee.servlet.QuartzInitializerServlet</servlet-class> <load-on-startup>1</load-on-startup> </servlet>
Job, you should pass the org.quartz.ee.ejb.EJBInvokerJob class to your JobDetail. To demonstrate this technique, we will implement ReportGenerator as a session bean and call its generateReport() method from the servlet.package net.nighttale.scheduling.ee;
import java.io.IOException;
import javax.servlet.*;
import net.nighttale.scheduling.Listener;
import org.quartz.*;
import org.quartz.ee.ejb.EJBInvokerJob;
import org.quartz.impl.StdSchedulerFactory;
public class ReportServlet implements Servlet {
public void init(ServletConfig conf)
throws ServletException {
JobDetail jobDetail = new JobDetail(
"Income Report",
"Report Generation",
EJBInvokerJob.class
);
jobDetail.getJobDataMap().put(
"ejb",
"java:comp/env/ejb/Report"
);
jobDetail.getJobDataMap().put(
"method",
"generateReport"
);
Object[] args = new Object[0];
jobDetail.getJobDataMap().put("args", args);
CronTrigger trigger = new CronTrigger(
"Income Report",
"Report Generation"
);
try {
trigger.setCronExpression(
"0 0 12 ? * SUN"
);
Scheduler sched =
StdSchedulerFactory.getDefaultScheduler();
sched.addGlobalJobListener(new Listener());
sched.scheduleJob(jobDetail, trigger);
System.out.println(
trigger.getNextFireTime()
);
} catch (Exception e) {
e.printStackTrace();
}
}
public ServletConfig getServletConfig() {
return null;
}
public void service(ServletRequest request,
ServletResponse response)
throws ServletException, IOException {
}
public String getServletInfo() {
return null;
}
public void destroy() {
}
}ejb: The JNDI name of the enterprise bean.method: The actual method to be called.args: An array of objects to be passed as a method arguments.
<ejb-ref> <ejb-ref-name>ejb/Report</ejb-ref-name> <ejb-ref-type>Session</ejb-ref-type> <home>net.nighttale.scheduling.ee.ReportHome</home> <remote>net.nighttale.scheduling.ee.Report</remote> <ejb-link>ReportEJB</ejb-link> </ejb-ref>
-userThread switch.These are by no means all the enterprise features of Quartz, but it's a good start and you should look at Quartz's Javadocs for further questions.
Web Services
Quartz currently has no built-in support for being used as a web service, but you can find a plug-in that enables you to export the Scheduler interface through XML-RPC. Building an installation procedure is simple. To start, you have to extract the plug-in source into the Quartz source folder and rebuild it. Plug-ins rely on the Jakarta XML-RPC library, so you have to be sure that it is in the classpath. Next, add the following lines in the properties file.org.quartz.plugin.xmlrpc.class = org.quartz.plugins.xmlrpc.XmlRpcPlugin
org.quartz.plugin.xmlrpc.port = 8080Summary
We've looked at two ways to do scheduling in Java. Quartz is indeed a powerful library, but for simple requirements theTimer API can save the day, keeping you from putting unnecessary complexity into the system. You should think of using the Timer API in cases where you don't have a lot of tasks that need to be scheduled, and when their execution times are well-known (and unchangeable) in the design process. In these situations, you don't have to worry whether some tasks are lost because of a shutdown or crash. For more sophisticated needs, Quartz is an elegant scheduling solution. Of course, you can always make your own framework, based on the Timer, but why to bother when all that (and much more) already exist?One event worth noting is IBM and BEA's submission of JSR 236 titled "Timer for Application Servers". This specification is focused on creating an API for timers in managed environments where using the standard API is insufficient. You can find more details about the specification on IBM's developerWorks site, and the specification should be available for public review in spring.
No comments:
Post a Comment