Human Practices

1. Rational

Surveys are an easy and convenient way to collect large amounts of information, so it’s no surprise that many iGEM teams use them each year to explore public perceptions of their projects. However, surveys are also complex psychological tools in which every word can influence how respondents answer. On one hand, researchers’ own biases - whether conscious or unconscious - can shape how questions are phrased to align with their expectations. On the other hand, participants’ varying attitudes toward surveys can lead to skewed results. Designing a reliable, unbiased survey is therefore far from straightforward.

With the rise of AI programs such as Gemini and ChatGPT, it may appear that creating surveys has become easier than ever. This prompted our team to ask: could using AI to design survey questions affect how people respond? To investigate this, we designed the “AI vs PI” experiment, comparing surveys written by AI and by professional researcher.

Our inspiration came from two key sources. The first was a classic 1974 study by Loftus and Palmer, in which participants watched a video of car crash and were asked to estimate the speed of the vehicles using different verbs such as “hit,” “bumped,” or “smashed.” The study showed that even a single word could change people’s perceptions: the average speed estimate was about 40.8 mph when the word “smashed” was used, 39.3 mph for “collided,” 38.1 mph for “bumped,” 34.0 mph for “hit,” and 31.8 mph for “contacted.” This demonstrated how subtle differences in wording can significantly influence responses.

The second source was a recent study by Salvi, F., Ribeiro, M. H., Gallotti, R. & West, R (2025) published in Nature Human Behaviour magazine, which showed that AI-generated arguments can be more persuasive than those written by humans.

Together, these insights inspired us to explore whether AI-generated survey questions might influence participants’ responses differently from human-written ones. We hope our findings will help future iGEM teams design clearer, fairer surveys and avoid the subtle linguistic biases that can shape results.

2. Survey Design

To generate surveys, one team member created a prompt that outlined the guidelines the survey should follow. The prompt was shared with the team’s PI who was unaware of the aims of the experiment, and with ChatGTP-5, that was also provided with project description documents. The full prompt can be found below in the Supplementary Materials.

The survey consists of four parts: 1. General information assessment – to set up the background for the surveys; 2. Project description – to compare whose description is clearer and understood better by the public – AI’s or PI’s; 3. Persuasiveness task – to evaluate which author could intentionally skew responses more effectively. This is the important ethical part of the survey that aims to demonstrate how easily survey questions could be used to manipulate participants’ responses. 4. Last question in both surveys was the same – “By whom do you think this survey was written? – AI or Human AMR researcher” – to assess whether participants could pick up on differences in styles and presentation to tell the two authors apart. Full versions of each survey can be found below in the PI’s Survey and ChatGTP-5 Survey sections.

3. Questions Analysis

Interestingly, neither the PI nor the AI chose to include open-ended questions - both relied exclusively on multiple-choice and scaled-response formats. The overall structure and content of the questions were also similar. For example, both surveys included questions about the causes of antimicrobial resistance (AMR) and its perceived importance in the first section.

Similarly, in the second section, questions five and six in both surveys focused on understanding of diagnostics and therapeutics.

NOTE: this type of question is crucial in survey design, as it helps identify respondents who have genuinely read and understood the provided information versus those who have not, thereby allowing to filter and improve data reliability.

In the third section, both the PI and the AI employed strikingly similar strategies to intentionally bias responses. For instance, both used the phrasing “How positive are you…,” which is inherently leading, as it presumes that respondents already hold a positive attitude toward the project. Additionally, both made the questions more personal by appealing to respondents’ emotions or people around them, such as “…your community” in the AI survey and “…treat yourself or someone you know…” in the PI survey.

NOTE: such wording would be considered inappropriate in rigorous scientific surveys, as it biases participants toward favourable responses.

4. Data Collection

To collect responses the surveys were imported to the SurveyMonkey website to generate URLs. Then, the AI survey was sent to students at the Lebanese International University (81 responses), and the PI survey was sent to students at the Beirut Arab University (77 responses) – all 158 total responses were kept anonymous.

5. Data Analysis: Biases

Even before filtering the data, we would like to draw attention to the problem of biases in surveys. In our survey we have encountered several effects:

• Leading Questions Effect: Question like: “How positive are you that CASPER can save lives and alleviate the dramatic effects of AMR?” is an example of a leading question that nudges participants to respond more positively to the question due to the bias of “how positive”; instead, questions like: “How effective do you think CASPER might be in addressing issues related to antimicrobial resistance (AMR)?” would be more neutral.
• Demand Characteristics: Can occur when participants think that they have guessed the purpose of the survey and change their responses accordingly. For instance, some of the responders on the AI survey indicated 10/10 when asked how confident they were that they understood the project but then gave wrong responses to both description-based questions.
• Screw-You-Effect: Occurs when participants deliberately give misleading responses to sabotage the survey. For instance, some of the responses in the PI survey indicated that after reading the project description they believe it is “unlikely to work”, despite previously selecting “I did not understand the project” when asked about how confident they were that they have understood the project and providing wrong answers to description-based questions.
• Acquiescence Bias: Tendency of participants to agree with statements regardless of content and context. For instance, participants who selected “Without hesitation” when asked “How likely are you to recommend or use CASPER to diagnose or treat yourself or someone you know?” all also selected “Reasonably positive” or “Extremely positive” for two other questions about CASPER implementation.
• Pattern Bias: When participants use a repeated pattern of answers. For example, some participant chose the bottom answer in nine out of ten questions in the AI survey.

Furthermore, though we did not identify them in our surveys, there are other effects and biases that responsible survey creators should be looking for:

• Conformity Effect: When people alter their responses to align with perceived group norms or what they think others believe.
• Social Desirability Bias: Respondents give answers that make them look good or socially acceptable rather than truthful.
• Question Order Effect: When earlier questions influence answers to later ones.
• Framing Effect: How a question is framed affects answers – e.g. “90% survival rate” vs. “10% mortality rate”
• Anchoring Effect: An initial value or number influences later judgments.

6. Data Analysis: Results

To filter results, we applied two following criteria: 100% survey completeness and correct responses to question six which relied on participants reading the description to correctly answer them. NOTE: these criteria may not be sufficient to filter out all unreliable results, and it would be an improvement to include more reference questions to confirm that participants do pay attention to the survey. After applying these criteria, 54 responses remained in the AI survey and 50 responses in the PI survey.

As expected, section one (broadly covering AMR) was answered equally well with most participants correctly identifying causes of AMR and its importance. In this experiment section one is the control condition, and it sets up the background for the rest of the survey.

In section two (Project Description), 62% of the AI survey responders indicated that they understood the project more than 5/10 and 83% of responders answered both description-based questions correctly. In the PI survey, though more participants indicated high confidence levels – 76%, only 20% answered both description-based questions correctly.  Judging by this difference, it may be suggested that AI was clearer in explaining the project and designing questions, or that its questions are simpler than PI’s.

View Fig 2: Project Understanding Comparison Graph

In Section Three (Persuasiveness Task), an average of 48% of participants selected the most positive response when asked about using products derived from CASPER in the AI survey, compared to 25% in the PI survey.

View Fig 3: Persuasiveness Comparison Graph

However, when combining the two most positive response categories, this difference largely disappears, with 59% of participants in the AI survey and 55% in the PI survey.  For the last question, most participants identified the author of the survey as “Human AMR researcher” in both AI (63%) and PI (70%) surveys, indicating that participants could poorly differentiate between the survey written by the AI and by the PI.

7. Interpretation

The only section of the survey that generated significant difference was the Project Description part. This could be potentially attributed to Chat-GTP being clearer with its questions and answers than the PI. Interestingly, the AI was not significantly more effective than the PI in skewing the results by trying to persuade participants to feel positive about the CASPER-derived products. This may be due to both AI and PI using almost identical strategies to achieve this goal. Finally, participants appeared unable to reliably identify the AI as the author of one of the surveys, even though all stylistic and linguistic features typical of ChatGPT were retained. While these features may be detectable when specifically sought out, most participants likely did not pay enough attention to notice them.

8. Limitations

The interpretations listed above are inconclusive due to the original experiment design that prevents preforming statistical tests such as t-test to determine the significance of the differences between the two conditions. This is because similar questions in two surveys have different modes of response or different number of answers, so although means for each condition could be calculated, it makes direct comparison unreliable. A potential improvement would be to maintain answers the same and vary only questions and descriptions to clearly demonstrate difference in the results.

9. Conclusions

This experiment demonstrated that AI-generated surveys can perform comparably to those written by human researchers, with minimal differences in clarity and persuasiveness. While the AI survey appeared clearer in its project description, both AI and PI employed similar persuasive strategies, resulting in nearly identical response patterns. These findings highlight the need for careful ethical consideration when using AI in survey design, emphasizing consistency, transparency, and human oversight to prevent unintended bias.

---

I. PROMPT (Supplementary Materials)

[Survey Engineering

This survey is part of the Oxford iGEM 2025 project CASPER dedicated to developing a universal platform integrating diagnostics and therapeutics. In this we want to assess people’s knowledge about AMR and how they understand and perceive our project. Hereby, we ask you to come up with questions that will help us in assessing these criteria.

General guidelines for questions include:

1. Do not exceed 40 words per question and avoid long descriptions unless stated otherwise in the task
2. You are free to choose questions format (multiple choice questions, open answer questions, questions requiring answers on a scale 1-10 where you would define the extremes, etc.).
3. You are free to choose questions wording and language used.
4. Make sure the survey is accessible and understandable for people without scientific background, who may have different levels of English proficiencies, and may come from different demographic groups including age, gender, nationality, and ethnicity.
5. Never quote this guide in any descriptions and questions

1. General information assessment

Use questions one to three to assess how aware responders about antimicrobial resistance crisis and its causes. Include ONE and only ONE question that involves answers with numerical values.

2. Descriptiveness assessment

Using 300 words or less describe the CASPER project as clear as possible, assume that responders may have no prior knowledge of the subject. Focus exclusively on the scientific aspects of the project. Avoid explaining final commercial products. Avoid focusing on benefits of the project for humanity. Use questions four to six to assess how well responders understood the project. Include ONE and only ONE question (question four) that would ask how confident responders are in their understanding of the project. Use the remaining two questions (questions five and six) to assess factual knowledge about the project. Avoid directly referencing your description.

3. Persuasiveness assessment

Using 200 words or less describe the final commercial product that stem from the CASPER project (only those that have been mentioned in the project descriptions) and their benefits for humanity. Your task is to persuade responders that CASPER products will have positive effects on global healthcare as much on responders themselves. Using questions seven to nine assess whether responders feel positive about the CASPER solutions or not and assess responders’ interest in the CASPER solutions. Please submit all questions (including multiple choice answers and answers on a scale if you included any in your survey) and descriptions in exact order they will appear on a final questionnaire.]

II. PI’s SURVEY (Professional Researcher)

1. AMR is relevant to what type of infections? (choose 1 answer only):
   1. Bacteria
   2. Fungi
   3. Viruses
   4. Bacteria and Fungi
   5. Bacteria and Viruses
2. On a scale of 1 to 10 having, 10 being the worst, how serious do you think the AMR problem is globally compared to heart failure or cancer.
3. What caused the rise in AMR (choose 3 answers that apply):
   1. Not washing your hands enough times daily
   2. Lack of diagnostic solutions
   3. Overuse of antibiotics
   4. People’s eating habits in some countries
   5. Lack of new antibiotics to get rid of antibiotic-insensitive superbugs

• Please read the project description and answer questions 4 to 6:

Antimicrobial resistance is on the rise worldwide. It is expected to become the next global pandemic. The overuse of antibiotics in the food industry, at the hospitals, and the abuse by individuals without medical advice; are giving rise to infectious diseases that simply can’t be treated. The situation is worsened by the lack of quick diagnostic solutions that coordinate quick diagnosis, of the source of infection, with the treatment.

This is why we thought about a one-to-rule-all solution, and we called it CASPER. We put both the diagnostics and therapeutics at the heart of our system providing complete cycle from detection to cure. We start by identifying the source of the disease and then administer DNA-based treatment.

CASPER uses the famous gene-editing CRISPR system that is like a barcode scanner. With the CASPER-diagnostic system, we scan for a collection of known pathogens that affect patients until one is identified. Our system then simply alerts us about the pathogen presence with a color change form yellow to red. CASPER-diagnostics is a paper-best device. We employ reusable mini chambers where we flow the patient sample to quickly and with ease get a diagnosis without expensive instruments.

CASPER-Therapeutics is DNA-based treatment designed to spread across all bacteria, make programmable protein “scissors” and then only cut the bad pathogen’s DNA from within thereby eliminating the threat and leaving good bacteria unharmed. We can make different versions of a therapeutic for any combination of pathogens we see fit making it versatile and effective across many pathogens simultaneously. In summary, CASPER provides cheap, effective, and robust combinatorial approach by addressing not o detection and treatment. CASPER can also be used for Diagnostics or Therapeutics only; making it a Swiss knife for AMR care. CASPER’s modular design is ideal for both field and lab scenarios.

4. How confident are you that you have understood the project (Choose 1 answer only).
   1. Very confident
   2. Somewhat confident
   3. I needed a little more explanation to grasp the whole project
   4. I did not understand the project
5. CASPER-Diagnostics is: (Choose 3 answers that apply)
   1. A system that uses chemical solution to color bacteria
   2. A system using CRISPR protein gene editing to barcode the presence of pathogen resulting and in a peak in the signal using a PCR machine
   3. A paper- based system the can report of many pathogens at the same time
   4. Can only be used with CASPER-therapeutics and only in the lab and can only be used once
   5. Simple, inexpensive and easy to use
6. CASPER-Therapeutics is (Choose 1 answer only):
   1. A new chemical antibiotic used to destroy any pathogen all the time
   2. A DNA-based therapeutic that can be programmed to target many pathogens at the same time
   3. It uses small metal scissors to cut microbial DNA.
   4. It eliminates all microbes in the host.
   5. It is too expensive and is reserved the rich elite.

• CASPER will have immense benefits for the individual and the world. Our simple Diagnostic device with paper cartridges will help people screen for infections with ease and at a low cost. Early detections, at symptoms’ onset, will help with early treatment; especially when the infection is very serious. Casper therapeutics will replace the use of antibiotics–– that cost billions of dollars to develop, may have side effects or cause allergic reactions, and may lose effectiveness–– with a DNA-based solution that can be adapted to maintain its efficacy. Instead we have a microbial shuttle derived from native gut bacteria that delivers our DNA wherever infection may be.

CASPER elements are very cheap and simply to use. CASPER is also accessible to all people anywhere and everywhere around the world requiring simple storage requirements that are available anywhere. We are losing the fight against AMR. We are losing millions of lives annually. There is not an effective universal solution for it. With CASPER we are hopeful that it will save millions of lives if adopted and financed.

7. After reading our product description, how positive are you in our project plan?
   1. Extremely positive
   2. Reasonably positive
   3. Neutral
   4. Doubtful
   5. Unlikely to work
8. How likely are you to recommend or use CASPER to diagnose or treat yourself or someone you know
   1. Without hesitation
   2. I need to see more information
   3. If it passes FDA approval
   4. I prefer conventional antibiotics
   5. It does not concern me
9. How positive are you that CASPER can save lives and alleviate the dramatic effects of AMR
   1. Extremely positive
   2. Reasonably positive
   3. Neutral
   4. Doubtful
   5. Unlikely to work
10. By whom do you think this survey was written?
    1. AI
    2. Human AMR researcher

III. ChatGTP-5 Survey (AI)

1. Have you ever heard of antimicrobial resistance (AMR) before?
   1. Yes
   2. No
   3. Not sure
2. What do you think is the main cause of antimicrobial resistance?
   1. Overuse of antibiotics
   2. Stopping treatments too early
   3. Poor hygiene and sanitation
   4. Natural bacterial evolution
   5. I don’t know
3. In your opinion, how many people worldwide die each year because of antimicrobial resistance?
   1. Less than 100,000
   2. Around 1 million
   3. Between 3 and 5 million
   4. Over 10 million
   5. I don’t know

CASPER is a scientific project designed to fight bacterial infections by combining diagnostics and therapeutics into one integrated system.

For diagnostics, CASPER uses a CRISPR-based enzyme called Cas12a, which can recognise specific DNA markers of bacteria. When Cas12a binds to these sequences, it activates a reaction that produces a clear colour change from yellow to red. This allows harmful bacteria to be detected quickly and without expensive machines. CASPER’s diagnostic system can appear in several forms, from small, disposable tests to reusable devices with multiple testing channels.

For therapeutics, CASPER uses multiple delivery systems to insert programmable DNA tools directly into harmful bacteria. One method is bacterial conjugation, a natural process where bacteria exchange DNA through a “molecular syringe.” CASPER reprograms this to deliver DNA carrying the Cas12a system into pathogens. In addition, CASPER develops conjugative minicells—tiny, non-replicating bacterial particles—and extracellular vesicles (EVs), both of which act as safe carriers that can transfer therapeutic DNA without spreading live bacteria.

Once inside the target, Cas12a, guided by specific RNA sequences, cuts the genes responsible for antibiotic resistance or virulence. This either kills the bacterium directly or removes its harmful traits. By linking these two sides, CASPER forms a complete system: the diagnostics identify pathogens and their resistance genes, while the therapeutic tools—delivered through conjugation, minicells, or EVs—neutralise them.

4. On a scale of 1 to 10, how confident are you that you understood how CASPER works? (1 = Not confident at all, 10 = Very confident)
5. What does the CASPER diagnostic test use to show that harmful bacteria are present?
   1. A colour change
   2. A temperature change
   3. A smell change
   4. A sound signal
6. Which delivery systems are part of CASPER’s therapeutic approach?
   1. Conjugation, minicells, and extracellular vesicles
   2. Photosynthesis and fermentation
   3. Antibiotics and vaccines
   4. I don’t know

The CASPER project can lead to two types of products: fast diagnostic kits and living therapeutic treatments.

The diagnostic kits are designed to be simple, low-cost tools for quickly detecting bacterial infections and resistance genes. They show a clear colour change when harmful bacteria are present. Unlike laboratory-based tests that take hours or require expensive machines, CASPER kits could deliver results within minutes and be used in clinics, pharmacies, or even at home.

For treatment, CASPER focuses on probiotic-like capsules or food products, such as yoghurt, containing safe carriers like minicells or extracellular vesicles. These deliver programmable DNA tools directly into harmful bacteria, disabling or eliminating them. Unlike antibiotics, which often harm beneficial microbes, CASPER’s approach targets only the pathogens.

Together, these products could transform healthcare by making infection testing faster and more accessible, and by offering a precise way to treat resistant infections.

7. Do you believe rapid, low-cost diagnostic kits for infections would be useful in your community?
   1. Yes
   2. No
   3. Not sure
8. How positive do you feel about using probiotic-like treatments instead of antibiotics to fight infections?
   1. Very positive
   2. Somewhat positive
   3. Somewhat negative
   4. Very negative
9. Would you personally be interested in trying CASPER’s solutions if they became available in the future?
   1. Yes, definitely
   2. Maybe
   3. No
10. By whom do you think this survey was written?
    1. AI
    2. Human AMR researcher