A recent study by 1Password reveals that AI-generated patches for vulnerabilities are effective only about 46% of the time, often leading to new issues rather than resolving the original flaws. This study, which examined 6,080 patches produced by two AI models, highlighted the AI's inability to consistently deliver reliable fixes, with many patches introducing additional vulnerabilities.
The research categorized outcomes into five groups, revealing that most patches either failed to fix the vulnerabilities or altered application behavior. Experts stress the importance of human verification for AI-generated code, advocating for functional and regression testing before integration. While AI holds potential for improving coding efficiency, the necessity for human oversight remains critical to ensure security and reliability.