Projecting CyberPatriot Outcomes

To view the live projections, go directly to The Magi.

CyberPatriot, the National Youth Cyber Education Program, hosts a new season of its cyber defense competition each year. For the last couple years, I have taken score data that they release publicly and projected outcomes for the open division in terms of advancement. With that system now publicly available and online, it’s worth discussing factors to think of when looking at the projection data.

Competition Changes #

In previous CyberPatriot competitions, the advancement from the first round to the National Finals competition was (with minor variations from year to year) as follows:

• Teams would compete in Round 1 and Round 2. Those scores would be combined, and a given number of top teams would advance to the semifinals.
• Teams who did not advance to Semifinals would be given a state (and regional?) recognition round. These teams would not progress to the National Finals competition.
• Teams in the Semifinals would compete, and the top 12 teams would progress to the National Finals competition.

Now, the progression works as follows (again, for open division):

• Teams still compete in Rounds 1 and 2, and scores are still combined. However, the top 30% of teams are judged to be in the platinum tier, 40% in the gold tier, and the bottom 30% in the silver tier.
• Teams in all tiers compete in state recognition rounds, however, only the top 3 teams from each state (and 12 wildcards) advance to the regional recognition round.
• The top 12 teams in the platinum tier’s regional recognition round advance to the National FInals competition.

With these changes, CyberPatriot allows more winners to get recognition (for winning their respective state and regional rounds in all tiers), but also immediately cuts teams from having the chance to proceed to nationals if not immediately placed in the platinum tier at the start. Failing to be placed in the platinum tier results in no advancement from any previous tier to platinum, nor any form of advancement to the national finals competition. The projections I make specifically target the path to Nationals, but it should be fairly easy to adapt them for both gold and silver tiers later on.

The Magi System #

The Magi is live and scraping the official CyberPatriot results (currently at 54.243.195.23) every minute during competitions. It is open source (GPLv3) and documented well enough for CyberPatriot 7 Round 2 projections only (at this time of writing, the round in progress).

Decisions #

• Only public data is used. While some have access to CyberPatriot’s Volunteer Central, because the data is not public, none of it is used in projecting scores.
• The all service division is not calculated. Unfortunately, its advancement system requires knowledge of each team’s category, which is not publicly available (top 30% of Civil Air Patrol, for example, is unknown as no teams are listed officially as CAP on the score page).
• The number of platinum slots is dependent on the current teams participating in either Round 1 or Round 2. All subsequent calculations will be made based on participation in these two rounds.
• Teams that have registered, but who have competed in neither round are not factored in (meaning that the number of teams going to platinum is not entirely accurate).
• Teams with rule violations (such as time exceeded and multiple instances open) are also not factored in or out of platinum. In the previously released official score data, several teams that had warnings for time were not penalized in score. The process for determining what constitutes a rule violation is not public, and as a result, the status of teams with time violations is up in the air. Teams with multiple instances of an image open are more likely to be penalized (as it means concurrent point gains are happening), but again, the process for resolving disputes is not public.
• No ties are resolved. The process that CyberPatriot uses to break a tie for the last open slots for platinum is not documented, and may have several factors in play (keeping in mind the possible factors, including: penalties, completion time, or a catch-all).
• The Cisco Networking Challenge is not factored in. The score data for it is, once again, private to the team who submits the score and CyberPatriot.

Trends #

• At this time of writing, no team in Round 2 has a perfect score (300/300) vulnerabilities. Typically, at least some (although a very small number of) teams successfully complete both Round 1 and Round 2 images in their entirety.
• Team scores in Round 1 do not play as large of an impact as Round 2 scores. Round 2’s 300 point maximum (excluding Cisco) weighs performance in Round 2 as 60% of the final score. Furthermore, teams such as 07-0887, 07-1689 were able to score significantly lower Round 1 scores than their counterparts and still perform well enough to have a seat in the predicted platinum tier.
• No team that skipped Round 1 has scored high enough to qualify for platinum tier.
• With the current number of teams and the number of teams above a 300 score, skipping Round 2 will cause a team to not score high enough in total to qualify for platinum.