In the event that you missed the changes in ColdFusion 10, I will recap one of them. The scheduled task engine was replaced with the Quartz Job Scheduler. This fancy new scheduler comes with more features then you will probably ever use. There are now handlers for when tasks start, finish, and error. Tasks can now call other tasks when they finish. You can even define rules for when a task fails or fails to start on-time.

There is also greater control on when tasks run. You can now schedule tasks to run faster than every 60 seconds. You can also use crontime to schedule a task. For example, this... "0 23 ? * MON-FRI" states to run a task every weekday at 23:00:00. You could even do something crazy like this "2-59/3 1,9,22 11-26 1-6 ? 2003" which translates into "In 2003 on the 11th to 26th of each month from January to June every third minute starting from 2 past 1am, 9am and 10pm".

But in creating crazy schedules or faster run times you now run into another feature of the new scheduled task engine, misfires. Misfires are a fancy name for a task that didn't start when it should have. For example, if the task was scheduled to start at 10:00:00 and it is now 10:00:01 and the task has not started it is now considered misfired.

Missing the start time is just one of a few reasons a task can misfire. Some of the others are, the scheduler engine is down, there were no available worker threads, or a task was configured to start in the past. You can also generate a misfire by resuming a paused task. By default the scheduler uses a "smart policy" to handle misfires.

The "Smart Policy" is a set of rules on how the scheduler will handle misfires. Depending on how the misfire happens, they are handled in different ways. Here are a few examples...

Example 1: Task is created at 12:05:00 to start at 12:00:00 with no repeat. So, at the time of the task creation it already late. This condition could also occur if the scheduler engine was not running and it then started at 12:05:00 and discovered a late task. The "smart policy" for this condition is to fire the task immediately.

Example 2: Task set run every hour from 3:00pm - 5pm and repeat infinitely. However, it is now 4:10 and the task has not fired. The default policy is to ignore the misfires and get back on schedule. This means that the 3 and 4pm firings were missed and the task will fire at 5pm.

But, with all this there is a couple other factors that determine what to do on a misfire. The first is the setting in the ColdFusion Administrator. Each schedule task can be configured as to what to do when a misfire occurs. The default is to ignore the misfire. This would be equivalent to what the "Smart Policy" did for example 2 above. But you can could instead configure the task to fire a handler or, fire now.

But, before this all this misfire handling occurs there is one other, very important, setting that comes into play. There is a setting that determines that a task has truly misfired. But, this setting is nowhere in the Administrator. It is actually in the quarts.properties file. This file is found in the {cfinstallroot}/{instancename}/lib/quartz/ . Warning: Don't edit this file unless you know the ramifications of doing so. This file contains the advanced configuration settings for the Quarts engine. If you change the properties file you will need to restart ColdFusion for the changes to take effect.

This properties file contains a setting called "org.quartz.jobStore.misfireThreshold". This should be set to a value of 60000. This value is in milliseconds so it translates to 60 seconds. This setting is very key for how the quartz engine determine a misfire. A misfire is a task that has not fired on its expected start time. However, for the "smart policy" to be invoked the misfire must be classified as a task that failed to start and the difference between expected start and current time is greater than the misfireThreshold.

So, if a task was set to fire at 12:00:00 and it is currently 12:00:40 the task has misfired but the "smart policy" will not be invoked. Instead the task will be fired immediately. The task will not be flagged as a misfire and no misfire handler will be called. This is to account for expected delays in system overhead or there slight timing issues.

What if your task was scheduled to run every 5 seconds? What happens when it doesn't fire? This is where the misfireThreshold will get you into trouble.

Example 3: Task is set to start at 02:00:00 and fire every 5 seconds till 03:00:00. This would equate to 12 fires a minute over the next hour. It is now 02:24:00 and the task has been running for 24 minutes without issue. Let say the scheduler has an issue and fails for 30 seconds. It is now 02:24:30 and there have been 6 failed fires of the task. We are currently only 30 seconds since last task fire.

So, how would Quarts handle Example 3? Unfortunately, or fortunately, here is what happens. At 02:24:30 the engine notices that the next fire time of the task is in the past. It then checks the time gap and compares it to the misfireThreshold. It then notices that the time is under the threshold so the task is fired. This continues till the task is caught up. Thus immediately firing the task 6 times. No misfire handler was called nor did the task every fall into a misfire condition.

This same condition can occur if the task from example 3 was paused for 30 seconds and then resumed. However, this condition bothers me. I would say that if a task is paused then there should be no misfire. On resume of the task it should reschedule itself and not misfire. This is a behavior that I would like to see changed in a future release.

For the quartz engine to handle a task such as the one in example 3 and actually use a misfire handler the misfireThreshold needs to be lowered. It is suggested that the misfireThreshold be set to half of the smallest interval for all scheduled tasks. However, in lowering it can lead to unexpected side effects of other tasks now getting caught up in misfire conditions when they shouldn't have been.

If you set the misfireThreshold to be less than the default it is possible that you will generate more misfires then expected. This is where you will have to make intelligent decisions for the misfire setting on each task.

I hope this sheds some light on scheduled tasks and misfires. There are many more scenarios that can cause misfires so if you are using any advanced scheduling be on the look out for these.

Till next time,

--Dave