Millions of students use AI chatbots every day for homework help and even emotional support. Many don't know where their conversations and personal data end up afterward.
That question is now driving a new push for privacy rules in schools. Microsoft announced last week it agreed to adopt AI safety and privacy standards for its products used by students.
The company reached the deal with the American Federation of Teachers, the nation's second largest teachers union. The agreement is legally binding and includes third party audits to check compliance.
What the Agreement Covers
Under the deal, Microsoft will not use student or educator data to train its AI systems. There is one narrow exception for data that could help protect students from harm.
The company also says it will not sell student data or use it for advertising or product development. Families will get plain language explanations of how the tools actually work.
The agreement also bans AI companions or features built to create emotional attachment or dependency in students. It blocks tools designed to keep kids engaged longer than a task requires.
Brad Smith, Microsoft's vice chair and president, said the standards will apply to every school district the company has contracts with. That change takes effect November 1.
AFT President Randi Weingarten called it the first agreement of its kind. She said it fills a gap left by the absence of federal or state laws on AI in schools.
Other Tech Companies Respond
OpenAI and Anthropic both said they are talking with the AFT about similar agreements. OpenAI called the Microsoft deal an important step for AI safety and privacy in schools.
Anthropic said it supports efforts toward responsible AI use in education. The company said it is working with the AFT on what it called a gold standard for safety.
Google, which provides education technology used by 150 million teachers and students through Google Classroom, has not said if it will adopt similar rules. The company has not responded to requests for comment.
Google faced criticism in August after turning on its Gemini chatbot for many K-12 students inside Google Classroom. Before that, Gemini access had been limited to students 18 and older.
Meanwhile, the two largest school districts in the country are taking a more cautious path. New York City and Los Angeles both announced a one year pause on student AI use this school year.
Both districts plan to audit their education technology contracts during the pause. They are focusing on data privacy, transparency and accountability before expanding AI tools further.
Privacy experts say the Microsoft agreement could shape conversations between schools and tech companies going forward. Elizabeth Laird of the Center for Democracy and Technology said it may reduce the negotiating gap between wealthy and poorer school districts.
Josh Golin of the nonprofit Fairplay said the safety provisions are meaningful. But he cautioned that schools should not assume privacy protections mean AI tools belong in the classroom at all.
Weingarten said she personally contacted Google about the agreement before it was announced. As of now, the company has not said whether it will sign on.
