A few thoughts on testing from the trench

Yesterday ,in my coding process ,well,i run into a intricate problem . What i'm wanna do is to add a testcase for newly modified funcationality ,then verify whether it works or not. Since the functionality is not our concern , i'll leave out its nuts and bolts.The quick peek  on this is that it will read value of specific key from a web.config ,some sort of xml configuration file, then assemble that value to a legitimate datetime. So,quite simply,ha. Here comes the testcase that i added:

[Test]
public void should_return_true_when_time_frame_is_Null_and_current_time_passed_it()
{
   var timeFrame = DateTimeHelper.NULL_DATE_TIME;
   Assert.IsTrue(DateTimeHelper.TimeFrameLateThan(timeFrame, DateTimeHelper.DEFAULT_TIME_FRAME.AddMinutes(-1)));
}

As u may infer from foregoing code list , the TimeFrameLateThan method is the dirty worker that encapsulate the details. To clearly express that my emphasis on this ,  the TimeFrameLateThan will read value of key ,like "LatestCalculationTime" or whatever, and if that key is not configured in xml file ,then it simply use the default value ,in this case  "NULL_DATE_TIME" . To fully explain what's going on ,i need post a littile more context to u. Here is big picture:

[Test]
public void should_return_configed_timeframe_when_timeframe_in_app_setting_is_in_right_format()
{
   ConfigurationManager.AppSettings.Set(DateTimeHelper.TRIGGER_TIMEFRAME_KEY, "17:00:00");
   actualReturned = DateTimeHelper.ReadTimeframeFromConfig();
   expectedReturn = CreateUtcDateTimeByHHMM(17, 0);
   Assert.AreEqual(DateTimeKind.Utc, actualReturned.Kind);
   Assert.AreEqual(expectedReturn, actualReturned);
}

public void should_return_configed_timeframe_when_timeframe_in_app_setting_is_in_bad_format()
{
   ConfigurationManager.AppSettings.Set(DateTimeHelper.TRIGGER_TIMEFRAME_KEY, "15:70:00");
   actualReturned = DateTimeHelper.ReadTimeframeFromConfig();
   Assert.AreEqual(DateTimeKind.Utc, actualReturned.Kind);
   Assert.AreEqual(DateTimeHelper.DEFAULT_TIME_FRAME, actualReturned);

   ConfigurationManager.AppSettings.Set(DateTimeHelper.TRIGGER_TIMEFRAME_KEY, null);
   var configedTime = ConfigurationManager.AppSettings.Get(DateTimeHelper.TRIGGER_TIMEFRAME_KEY);
   Assert.IsNull(configedTime);
}

[Test]
public void should_return_true_when_time_frame_is_Null_and_current_time_passed_it()
{
   var timeFrame = DateTimeHelper.NULL_DATE_TIME;
   Assert.IsTrue(DateTimeHelper.TimeFrameLateThan(timeFrame, DateTimeHelper.DEFAULT_TIME_FRAME.AddMinutes(-1)));
}

Maybe now ,u ,as acute and perceptive as always ,have already seen through the problem. Well , at the time i coded it , i just did not got it. When  i run this test in IDE , it just happily gave me a green bar,hoho ,not bad. But when i run this from console (triggered by Nant ) , it just kept telling me that test--public void should_return_true_when_time_frame_is_Null_and_current_time_passed_it, failed.  Returned to the test codes,i just pondered why it just happened.  Here is how i spot the "tricky insect" in tests : first just remove all ,simply leave only failed testcase alone,then running the tests it passed.  Secondly pulled back testcase one by one ,simply follow the rhythm ---"add one  testcase,run the test ;if ok,repeat that" .  Finally  thing  that if i run  testcase 1 and test3 together ,it would break;but if i run test2 and test3 together ,it just passed. So i checked distinction between testcase1 and testcase 2 .Soon i figured that out , simply by some stupid mistake   , i forgot to set DateTimeHelper.TRIGGER_TIMEFRAME_KEY to null. It seems that i just forget do  some clean stuff,while what goes beyond the surface is that  I do not much grip test principle. When comes to test,it is so important to keep it self-stand and self-closed ,make sure it will not pollute and break other test case 's preconditions/assumptions. Here is the solution(I add some cleanup code between line 10 and line 13):

[Test]
public void should_return_configed_timeframe_when_timeframe_in_app_setting_is_in_right_format()
{
   ConfigurationManager.AppSettings.Set(DateTimeHelper.TRIGGER_TIMEFRAME_KEY, "17:00:00");
   actualReturned = DateTimeHelper.ReadTimeframeFromConfig();
   expectedReturn = CreateUtcDateTimeByHHMM(17, 0);
   Assert.AreEqual(DateTimeKind.Utc, actualReturned.Kind);
   Assert.AreEqual(expectedReturn, actualReturned);

   ConfigurationManager.AppSettings.Set(DateTimeHelper.TRIGGER_TIMEFRAME_KEY, null);
   var configedTime = ConfigurationManager.AppSettings.Get(DateTimeHelper.TRIGGER_TIMEFRAME_KEY);
   Assert.IsNull(configedTime);
}

public void should_return_configed_timeframe_when_timeframe_in_app_setting_is_in_bad_format()
{
   ConfigurationManager.AppSettings.Set(DateTimeHelper.TRIGGER_TIMEFRAME_KEY, "15:70:00");
   actualReturned = DateTimeHelper.ReadTimeframeFromConfig();
   Assert.AreEqual(DateTimeKind.Utc, actualReturned.Kind);
   Assert.AreEqual(DateTimeHelper.DEFAULT_TIME_FRAME, actualReturned);

   ConfigurationManager.AppSettings.Set(DateTimeHelper.TRIGGER_TIMEFRAME_KEY, null);
   var configedTime = ConfigurationManager.AppSettings.Get(DateTimeHelper.TRIGGER_TIMEFRAME_KEY);
   Assert.IsNull(configedTime);
}

[Test]
public void should_return_true_when_time_frame_is_Null_and_current_time_passed_it()
{
   var timeFrame = DateTimeHelper.NULL_DATE_TIME;
   Assert.IsTrue(DateTimeHelper.TimeFrameLateThan(timeFrame,
DateTimeHelper.DEFAULT_TIME_FRAME.AddMinutes(-1)));
}

Keep test case responsible for what it has done and ensure the circumstance before and after is consistent. Sometime it is your duty, so you  account for restoring in the first place;well ,somehow the intellecture framework will take care it for u .Take Spring's AbstractTransactionalDbProviderSpringContextTests as an example ,Spring will automatically add  transaction support to your test case in Setup and Teardown,to rollback any changes made in your test case. Anyway,it is aways a good habit to  let a test case account for cleaning what it has done.