1
submitted 4 months ago by [email protected] to c/[email protected]

Subverting Betteridge's law of headlines. Yes.

top 2 comments
sorted by: hot top controversial new old
[-] [email protected] 1 points 1 month ago* (last edited 1 month ago)

Quoting the abstract (I added emphasis and paragraphs for readability):

AI code assistants have emerged as powerful tools that can aid in the software development life-cycle and can improve developer productivity. Unfortunately, such assistants have also been found to produce insecure code in lab environments, raising significant concerns about their usage in practice.

In this paper, we conduct a user study to examine how users interact with AI code assistants to solve a variety of security related tasks.

Overall, we find that participants who had access to an AI assistant wrote significantly less secure code than those without access to an assistant. Partici- pants with access to an AI assistant were also more likely to believe they wrote secure code, suggesting that such tools may lead users to be overconfident about security flaws in their code.

To better inform the design of future AI-based code assistants, we release our user-study apparatus and anonymized data to researchers seeking to build on our work at this link.

Caveat; quoting from section 7.2 Limitations:

One important limitation of our results is that our participant group consisted mainly of university students which likely do not represent the population that is most likely to use AI assistants (e.g. software developers) regularly.

[-] [email protected] 1 points 4 months ago

To me this seems obvious, the models are trained off of GitHub as a whole. Most code on GitHub either is unsecure, or it was written without needing to be secure.

I'm already getting pull requests from juniors trying to sneak in AI generated code without actually reading it.

this post was submitted on 16 Feb 2024
1 points (100.0% liked)

Programming

13266 readers
7 users here now

All things programming and coding related. Subcommunity of Technology.


This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 1 year ago
MODERATORS