BaseLine Batch can alert you to any security changes at any of the email addresses you send to. It is the third step in using CheckTLS to meet your email security needs:
Check your own email system.
Use the More Options features on the two tests on our home page, Check How You Send Email and Check How You Get Email to thoroughly test your email.
Check email addresses you send to.
Use CheckTLS "Batches" to run ("TestReceiver") on a number of email addresses and only report on a few details like ConfidenceFactor, TLS Version, etc.
Know if anything changes.
That last step is important because you are responsible for protecting mail you send (See "The Responsible Party" section of Internet Email Security Made Simple)
your emailer will send plain text if the recipient asks you to (prove this with ).
With BaseLine Batch, you run your Batch of email addresses once and tell CheckTLS to save the results. Then when you run it again, you only see the differences between today and the saved results (the "BaseLine").
BaseLine Batch makes checking email addresses 1000 times faster. Instead of periodically looking at every address, only look at the ones that have changed. A tuned BaseLine Batch typically flags one out of a thousand addresses.
At any time you can update a BaseLine score, so if a target fixes or improves their email, you can set the new score as their BaseLine.
With BaseLine Batch, you:
- list all the email addresses you send to (just the Domain part, the "Target")
- test the addresses and split the results into "Good", "Bad", and "Untestable" Batches
- run the Batches once to set the "BaseLine" scores
- change the Batches to compare BaseLines
- tune the Batches
- run them regularly
These steps are described below.
Using BaseLine Batch
We suggest you follow the steps below with raw XML files using the example Targets, but use Excel workbooks to work with your production tests.
1. list all the email addresses you send to
Use the New button in to create a Batch of all the targets you want to BaseLine:
In production, we recommend limiting a Batch to about 5000 Targets and using multiple Batches if you have more that.
2. test the addresses once and split the results
split the Targets into three new Batches:
- Download the csv results.
Open the csv in Excel and sort the results by Score:
- Invalid Targets that are not valid email domains so could never be sent to. Fix them (typos) or remove them.
Bad Targets (did not do TLS)
You should not be sending any confidential or protected information via email to these. Any email sent to them is going over the Internet in plain-text, which is likely illegal.
Put them in a Bad Batch. When/if one becomes secure and you start sending to it, you can move it to a Good Batch.
Unreachable Targets that we could not test.
These could be typos or intermittent failures (see Vagaries below).
Remove any invalid Targets. Move Targets that are consistently not testable to an Untestable Batch. Good Targets that are only infrequently unreachable should stay in the Good Batch.
Bad Targets (did weak TLS)
You should not be sending any confidential or protected information via email to these. Email sent to them is encrypted but the encryption is easily broken.
Move them to a different Batch (Bad) and test them on a different schedule. When/if one becomes secure and you start sending to it, you can move it to a Good Batch.
Good Targets (did good or very good TLS)
These email systems use good (TLS v1.2) or very good (TLS v1.3) encryption.
Test these frequently to be sure they stay secure and all email you send to them is safe and legal. When/if one becomes "Bad" you should stop sending any email to that domain and move it to your Bad Batch until it is fixed.
- If you have questions about a Target's score, use the interactive test to look at the details.
These Batches should look like this:
<BatchTest TestType="setbaseline"> <Target>CheckTLS.com</Target> <Target>Invalid.CheckTLS.com</Target> <Target>Refuse.CheckTLS.com</Target> <Target>TLSv1.CheckTLS.com</Target> <Delivery> <Format>csv</Format> </Delivery> </BatchTest>If you are working through this page to learn BaseLine Batch, we suggest you just create one Batch with all the example Targets for now.
- Note the TestType is "setbaseline" and the Format is csv.
3. run the Batches once to set the "BaseLine"
Run the Batches and wait for them to finish.
4. change the Batches to compare BaseLines
Switch the Batches from setting the BaseLine to checking the BaseLine by changing the TestType to "baseline":
Running the Batches now will compare the current state (Score) of each Target with the saved BaseLine state.
The result is just the targets that have changed:
You can also use <Format>xml</Format>:
Notice BaseLine test has a Match node so you do not have to parse and compare scores.
5. tune the Batches
Taking time to tune the BaseLine will save time in the future and improve your results.
If you are working through this page to learn BaseLine Batch, we suggest you use to change the BaseLine score for the example Targets and run the BaseLine Batch to see the effect. For example if you change the BaseLine score for TLSv1.CheckTLS.com to 100 ("good") and CheckTLS.com to 0 ("bad"), running the BaseLine Batch will show that TLSv1.CheckTLS.com was "good" and is now "bad" and CheckTLS.com was "bad" and is now "good".
Once you have run a BaseLine, the following tuning can improve your results:
- Remove Targets that frequently don't match their BaseLine by editing the Batch in .
- Allow several scores or score ranges to match in .
- Keep Targets in their proper Batch based on their type or purpose (see Types of BaseLines below).
- Removed Targets will be automatically removed from the BaseLine when you next run it.
- Infrequently unreachable (Score=50) Targets should be removed from the stored BaseLine Scores but not removed from the Batch. That way they will still test despite being intermittently untestable. Use the RangeDelete option in to remove all Targets with a score of 50.
- Changed Targets should be updated in . This where the matching scores are stored.
- If you make a lot of changes to the Batch you should re-run a "setbaseline" Batch to save a new BaseLine. Re-running a "setbaseline" Batch will remove all the old saved BaseLine scores and replace them with current scores.
For really large Batches (thousands of Targets), see the next section "BaseLine Batch Auto-Tune".
6. run them regularly
When you want to compare the present settings with your saved BaseLine, run the Batch. As above, the result will be just the targets that have changed.
Periodically, if not every time, you should tune the BaseLine with new results. If a Target changes you should remove the old Match and add the new one, so future BaseLines will BaseLine against the Target's new Score.
If you ever want to completely reload all the BaseLines from this Batch, just reset the Format value and add the BaseLine attribute back and run the Batch. Just don't forget to put them back or you will replace the BaseLine every run instead of comparing to the BaseLine.
Moving to Production
When you have your BaseLine Batches tuned to only report problems, you can tell the Batches to only send a results email if something changed: either a Production ("Good") Target going bad, or a bad Target getting fixed. See Suppress in About Batch.
BaseLine Batch Auto-Tune
Auto-Tune automates the process of tuning large Batches. It removes mis-matches (failed BaseLine Targets) by adding new allowed matching scores for the mis-matches.
This is similar to re-running the Batch from scratch to reset all the BaseLines, but instead of replacing all the scores it adds new ones for any Targets that need them.
You should review the results of an automatic tune to make sure you don't inadvertently add a score that you do not want to accept. Targets effectively disappear from your exception report when you add their Score as a match.
To Auto-Tune a BaseLine Batch:
- You must have run at least one BaseLine (i.e. a Batch with TestType set to "baseline" -- see above).
- You must format the result as xml (i.e. <Format>xml</Format> -- see above).
- Use the AutoTune button in to open the AutoTune screen.
The Minimum Score depends on the purpose of this BaseLine (see Types of BaseLines below):
- Good Email
- 88 and above (does good encryption)
- Bad Email
- 51 to 87 (does weak encryption)
- Untestable Email
- 50 (unreachable, therefore untested)
- Bad Email
- 0 to 49 (does not do encryption)
- Check the XML Online box. This will use the copy of your results that we have stored on our servers.
See below for how to manually fine tune the Auto-Tune.
- Click the RunTune button to perform the Auto-Tune.
Your screen will show a summary of the new BaseLine scores that Auto-Tune added in the Test Results section under the buttons:
added: weakdomain.com = 90 added: anotherweakdomain.com = 85; ... skipping duplicate: gooddomain.com = 90..100 added 274 records
- Next time you run the BaseLine, the mis-matches you had will be gone.
Manual BaseLine Batch Auto-Tune
To Manually Auto-Tune a BaseLine Batch:
Download the last BaseLine Batch results ().
Not the original run, but an xml run, i.e. one that looks like this:
<Result> <eMailAddress>weakdomain.com</eMailAddress> <BaseLineScore>60</BaseLineScore> <CurrentScore>90</CurrentScore> <Match>0</Match> </Result>
- Load the results file into Excel, sort by CurrentScore, and note any lines you don't want to update.
- Remove the records in an editor that understands XML (Excel will not let you save XML) or any text editor if you are careful to preserve the proper XML formatting.
- Use the AutoTune button in to upload your edited results file and then to run it.
Your screen will show a summary of the new BaseLine scores that Auto-Tune added in the Test Results section under the buttons:
added: weakdomain.com = 90 skipping duplicate: gooddomain.com = 90..100 added 1 record
- Next time you run the BaseLine, the mis-matches you specified will be gone.
Types of BaseLines
We suggest you group email addresses ("Targets") that you want to test into three types:
Remember that our Confidence Factor does not score the email address; rather it specifically scores the security of the email address.
A BaseLine of Good Targets is a list of email addresses that you use that must be encrypted. You want to know if ever any one of them "breaks" and stops accepting encrypted emails. If one does break, you know to stop using it and contact them (maybe before they even realize they have a problem). If the Target is broken for a long time you may have to move it to one of the below BaseLines.
These should have Match Scores of 90 or above. Why accept a score that is measurably insecure? If you do find a Target below 90 that you want to accept, we allow you to add it.
A BaseLine of Bad Targets is a list of email addresses that you are not using because they are not encrypted. You want to know if ever any one of them gets "fixed" and starts accepting encrypted emails. Once they are OK you can let your organization know they can start using these email addresses. Then you should move fixed addresses to a Good BaseLine to make sure it stays secure.
These have Match Scores of 0 (zero) to 49. Note 50 is a special case below.
A BaseLine of Untestable Targets is a list of email addresses that CheckTLS is unable to test. We don't know if they are secure or not, so we give them a "fifty fifty" chance. You want to know if ever any one of them gets "fixed" so CheckTLS can reach them and start testing them.
These have Match Scores of exactly 50.
Working Around Email Vagaries
Internet Email is designed to "never lose an email" and "get the mail through". It has enough redundancies to make sending any one email an adventure, as two emails sent at the same time can go two different ways. That makes it impossible to answer the question "Will every email ever sent to XYZ.com be properly encrypted?"
Which is why CheckTLS created the ConfidenceFactor. It takes everything into consideration to answer the question "How good is XYZ.com's email security?"
We call all the differences that an email system can exhibit "Vagaries".
Vagaries can result in a "false report" of a BaseLine change. The most common "false report" is when an email server gets busy and tells the sender to "try again later". This is expressly allowed by the formal email standard, and the sender is instructed to just try again in a few minutes. But our real-time test reports that as a failure to connect.
Other "false reports" come from one server in a redundant server pool having slightly different settings, or catching a site in the process of routine maintenance, etc. All these result in a slightly different Confidence Factor.
When doing BaseLine testing, we recommend TestType="receiverquick". "receiverquick" uses the Quick option in to test only the one most likely MX host for a domain. This eliminiates most Vagaries.
Between any two BaseLine "receiverquick" runs we see less than 0.5% mis-matched Confidence Factors. So CheckTLS BaseLine testing makes checking your trading partners' emails 200 times easier.
A few simple tune-ups can reduce false reports to one in a thousand or less:
Use to set the score or scores that are acceptable.
- Manually set the score
- Add two or more scores
- Add a range of scores
- Move any Targets with frequent "false positives" to a separate Batch, or remove them altogether. BaseLine doesn't work well for Targets that change all the time.
BaseLine testing has some additional features:
Multiple Scores and Score Minimums and Maximums
lets you list two or more scores that will match a Target. It also lets you set a scoring range that will match, so for example you can use the range 88 to 121 to mean "Yes this domain uses TLS 1.2 or better".
Multiple BaseLines for a Single Targets
The standard ("TestReceiver") test has many options.
BaseLine Batch lets you check the same Targets with different options.
For example, you can BaseLine the Targets's security with TLS version 1.2 and separately with TLS version 1.3:
This will show you when XYZ.com changes from TLS v1.2 to TLS v1.3.
BaseLine Batch keeps the full Target email address so you can BaseLine the same Targets with different settings.
BaseLine and Details in the Same Batch
With a simple hack you can use a BaseLine Batch to show details like SSLVersion and Cert expiration for the same Targets that are in the BaseLine.
Keep two <Delivery> nodes at the bottom of the Batch, one that makes it a BaseLine and one that makes it show details.
Comment out one or the other to control what the Batch does.
You can maintain the stored BaseLines for a Batch by using BaseLineEdit as a WebService.
- ADD A TARGET:
https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODEfirstname.lastname@example.org &CUSTOMERPASS=IllNeverTell &BATCHID=52 &TARGETemail@example.com &SCORE=99 &ACTION=wsAddTarget
<Results> <Result>Success: firstname.lastname@example.org(#52) = 99</Result> </Results>
Any errors returns:
<Errors> <Error>Invalid CUSTOMERCODE or CUSTOMERPASS: See FAQ</Error> <Error>Invalid BATCHID: 1a</Error> <Error>Invalid TARGET: (blank)</Error> <Error>Invalid SCORE: alpha</Error> </Errors>
- DELETE A TARGET:
https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODEemail@example.com &CUSTOMERPASS=IllNeverTell &BATCHID=52 &TARGETfirstname.lastname@example.org &ACTION=wsDeleteTarget
<Results> <Result>Success: 1 record deleted</Result> </Results>
- DELETE ALL TARGETS:
https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODEemail@example.com &CUSTOMERPASS=IllNeverTell &BATCHID=52 &ACTION=wsDeleteAllTargets
- DELETE BY SCORE:
Delete all records with a ScoreMin between 0 and 10 and a ScoreMax between 20 and 30:
https://www.CheckTLS.com/BaseLineEdit ?CUSTOMERCODEfirstname.lastname@example.org &CUSTOMERPASS=IllNeverTell &BATCHID=52 &SCOREMIN=0..10 &SCOREMAX=20..30 &ACTION=wsDeleteScores
To delete all records with a ScoreMin below 50 use:
<Results> <Result>Success: 437 records deleted</Result> </Results>