The State Department is using artificial intelligence to transform how it declassifies documents and processes Freedom of Information Act requests, resulting in major cost savings, a better customer experience and easier access to agency records at a time when most Americans view government as lacking in transparency and wasteful.  

At the center of this transformation is Eric Stein, deputy director in State’s Office of Management Strategy and Solutions, and a former participant in the Partnership’s AI Federal Leadership Program, an initiative supported by Google and Microsoft that prepares leaders in government to effectively and responsibly use AI.   

Using what he learned in the program, Stein worked with the agency’s Center for Analytics to launch three pilot programs over the past two years that declassified more than 78,000 diplomatic cables using machine learning and used new AI tools to accelerate the process for handling records inquiries and requesting State Department documents.  

“I understood AI very superficially before taking the course. Today, I know how to use AI ethically in programming, policymaking and executive decision-making,” Stein said.   

A New Pilot

To declassify the diplomatic cables, Stein and his team trained a machine learning model on previous human declassification decisions and then ran thousands of records through the new system from October 2022-January 2023.  

The machine’s decisions on whether to declassify documents turned out to be 97% as accurate as they were when manually reviewed. Stein said the AI system cost just $400,000 to develop, reduced staff time by around 60% and saved his office about half a year’s worth of work. 

Stein’s “aha!” moment for the pilot came during a session in the AI Federal Leadership Program on using AI to manage and share data. He said the course also taught him how to integrate human review and oversight into the AI system to ensure it makes correct decisions.  

“During the course, the facilitators were talking about large volumes of information, and I thought, ‘wait a minute, why are we spending so much time and energy on manually declassifying information?’ I saw the potential to make a connection between what I was learning and what I do professionally,” he said. 

Building on Success

The results of the pilot encouraged Stein and his team to go further.  

In June 2023, they used what they learned to begin using new AI tools to search for documents in response to Freedom of Information Act requests. The new system, with human assistance, can identify which records are publicly available and whether there have been previous searches for the same material, enabling staff to more quickly process the thousands of record requests State receives each year.  

Another pilot, launched around the same time, provides users with a new online platform to make a request and track the status of their inquiry. The platform also links to State’s online reading room, which features documents released under the Freedom of Information Act.  

“We are transforming how the agency makes information available to researchers and others in the field,” he said.  

The Work Continues

The Bureau of Administration, where Stein worked, is expanding its new declassification system to handle electronic records, including emails, which are more voluminous than older paper-based documents and will seriously stretch State’s manual review process. 

The team also helped establish State’s first agencywide AI policy guidelines and is briefing dozens of agencies, and the National Security Council, on its work.

“The course helped me consider the key elements that needed to be added,” he said.  

Taken together, these accomplishments make Stein a leader at the forefront of federal AI. What began as the seed of an idea planted in the AI Federal Leadership Program has blossomed into a core part of State’s latest advancements in data and technology.   

“The program changed my professional experience at State,” he said. “It shaped how we’re doing business and how we think about AI, and it was inspirational and motivational. The last few years would have been very different had I not enrolled.”