The FAA paused every US departure after files were deleted while fixing a database sync.

A NOTAM (Notice to Air Missions) warns pilots about things like closed runways, flocks of birds or volcanic ash. Late on 10 January 2023, the FAA's NOTAM system became unreliable.

Technical experts tried, among other things, switching to a backup database. There were three: one in Oklahoma City and two in Atlantic City. By early morning the system seemed restored, but formatting problems persisted, so the FAA's air traffic leadership ordered the databases rebuilt.

With the morning rush coming, acting FAA Administrator Billy Nolen ordered a ground stop at about 7:15 a.m. EST, pausing every departure in the United States. He later told senators he lifted it at 9:07 a.m., after resiliency testing. Senate Commerce chair Maria Cantwell put the toll at more than 1,300 cancelled flights and more than 9,500 delayed.

The FAA first traced the outage to "a damaged database file". On 19 January it said: "contract personnel unintentionally deleted files while working to correct synchronization between the live primary database and a backup database." It found no evidence of a cyber-attack.

Why didn't the backups help? At a Senate hearing on 15 February, Nolen said changes synchronise across the primary and the backups. The FAA's written answers for the record add that the deletion "did not disrupt synchronization": sync "had not been occurring with all secondary databases" before the outage.

Its fixes: a one-hour synchronisation delay between the primary and the backup database, "to ensure that bad data from a database cannot affect a backup database", and a rule that more than one person must be present when work is done on the database.

Why it matters: A replica that copies every change instantly also copies every mistake, and one that has quietly stopped syncing can't take over. A backup you can recover from needs a delay or a snapshot, and a check that it is current.