Seeking ChatGPT advice on questionnaire survey research design: a note for the part-time Housing Studies students

Seeking ChatGPT advice on questionnaire survey research design: a note for part-time Housing Studies students

I have had a conversation with the assistant bot (a ChatGPT) on poe.com about some concerns related to questionnaire survey practice from the part-time Housing Studies students. An account of the conversation is presented below.

Query 1

Joseph Ho (student): "I am a part-time Housing Studies student. My dissertation project involves doing a questionnaire survey on the public perceptions of the images of rough sleepers in Hong Kong. Due to resource constraint, I can only afford to employ convenience sampling for my survey research. Essentially, I am going to ask my friends and my friends' friends to participate in the survey. I thus have two questions to ask: (1). what are the limitations of using convenience sampling in this case, bearing in mind that most of my friends are middle-aged adults with tertiary education backgrounds and I have about 20 friends only and (2). how can I motivate my friends to also invite their friends to participate in my survey?".

Assistant bot: "Hello, I'm happy to provide some insights on your dissertation project and the questions you've raised.

Limitations of using convenience sampling in this case:

Representativeness: With a small network of friends, predominantly middle-aged adults with tertiary education backgrounds, the sample may not be representative of the general public's perceptions of rough sleepers in Hong Kong. This could limit the generalizability of your findings.

Sampling bias: Your friends and their networks may share similar socioeconomic and demographic characteristics, which could introduce bias into the data and skew the results.

Statistical validity: A small sample size of 20 friends and their friends may not provide enough statistical power to detect meaningful differences or draw robust conclusions about the public's perceptions.

Lack of diversity: The homogeneity of your sample may not capture the diverse perspectives and experiences of different segments of the Hong Kong population, such as younger adults, lower-income individuals, or those from different cultural backgrounds.

Strategies to motivate your friends to invite their friends to participate:

Emphasize the importance of the research: Clearly communicate the significance of your dissertation project and how their participation can contribute to a better understanding of the public's perceptions of rough sleepers in Hong Kong.

Offer incentives: Consider providing small incentives, such as gift cards or raffle prizes, to encourage your friends and their friends to participate. This can help increase the response rate.

Highlight the ease of participation: Make the survey process as straightforward and convenient as possible, such as by providing an online survey link or a mobile-friendly format.

Foster a sense of community: Encourage your friends to feel invested in the project by positioning it as a collaborative effort to gain insights that can inform policies or support services for rough sleepers.

Provide updates: Keep your friends informed about the progress of the study and share any preliminary findings or insights. This can help maintain their engagement and interest.

To address the limitations of convenience sampling, you may want to consider supplementing your primary data collection with other sampling methods, such as purposive sampling or snowball sampling, to reach a more diverse and representative group of participants. Additionally, you could explore ways to triangulate your findings, such as by incorporating secondary data or qualitative interviews, to provide a more comprehensive understanding of the public's perceptions.

I hope these suggestions are helpful for your dissertation project. Let me know if you have any other questions.".

Query 2

Joseph Ho (student): "I am also concerned that if I ask too many questions on my online survey questionnaire form, the participants are not motivated to complete the questionnaire. Are there any things I need to pay attention to so that the participants are more willing to answer all the questions on the questionnaire form?"

Assistant bot: "That's a valid concern. The length and structure of your survey questionnaire can have a significant impact on participation and completion rates. Here are some tips to consider:

Keep it concise: Aim to include only the essential questions needed to address your research objectives. Avoid unnecessary or redundant questions that could make the survey feel overly long and burdensome.

Use clear and simple language: Ensure that the wording of your questions is straightforward and easily understandable for your target participants. Avoid jargon or complex phrasing that could confuse or discourage respondents.

Organize the questionnaire logically: Group related questions together and use clear section headings or instructions to guide participants through the survey. This can help create a more coherent and user-friendly flow.

Limit the number of open-ended questions: While open-ended questions can provide rich qualitative data, they also require more effort from respondents. Try to balance open-ended questions with multiple-choice or scaled questions, which are generally quicker to answer.

Optimize the survey length: As a general rule, aim for a total completion time of 10-15 minutes or less. If your survey is significantly longer, consider breaking it into multiple sections or offering respondents the option to save their progress and complete the survey in multiple sittings.

Provide progress indicators: Include a progress bar or indication of how many questions remain to help participants understand the survey's length and their progress.

Offer incentives or rewards: As mentioned earlier, providing small incentives or the chance to win a prize can help motivate participants to complete the full survey.

Express gratitude: Acknowledge and thank participants for their time and contribution at the beginning and end of the survey.

Pilot test the survey: Conduct a small-scale pilot test with a few individuals from your target audience to gather feedback on the survey's length, clarity, and overall user experience. Use this feedback to refine the questionnaire before launching the full study.

By implementing these strategies, you can create a more engaging and user-friendly survey experience, which can help increase the likelihood of participants completing the full questionnaire".

Query 3

Joseph Ho (student): "I am also concerned that if the cut-off period of the questionnaire survey is too long, it will affect the quality of the survey data quality. My question is then: what would be an appropriate cut-off period of the survey to adopt?".

Assistant bot: "That's a valid concern. The length of the data collection period can indeed impact the quality and reliability of the survey data. Here are some factors to consider when determining an appropriate cut-off period for your survey:

Research objectives and timeline: The cut-off period should be aligned with the timeline and requirements of your dissertation project. Consider the deadlines you need to meet and the minimum amount of time required to collect a sufficient amount of data.

Participant availability and response rates: Evaluate how long it may take to reach your target sample size, accounting for factors like the availability of your convenience sample and their willingness to participate. A longer data collection period may be necessary if you anticipate lower response rates.

Potential changes in the research context: Hong Kong's social and economic landscape may evolve over time, and you want to ensure that the data you collect reflects the current public perceptions of rough sleepers. A shorter cut-off period can help minimize the risk of changes in the research context affecting your findings.

Data quality and consistency: A longer data collection period increases the likelihood of variations in participant responses due to factors like changing moods, experiences, or external events. A shorter cut-off period can help maintain data quality and consistency.

Considering these factors, a reasonable cut-off period for your survey might be around 4-6 weeks. This timeframe should provide enough time to reach your target sample size while minimizing the potential for changes in the research context and ensuring a higher degree of data quality and consistency.

If you find that you are struggling to reach your desired sample size within this timeframe, you may need to consider adjusting your recruitment strategies or extending the cut-off period slightly. However, aim to keep the data collection period as focused and compact as possible to maintain the integrity of your findings.

Remember, the specific cut-off period you choose should be informed by your research objectives, the characteristics of your target population, and any practical constraints you may face. Regularly review and evaluate the data collection progress to ensure you make informed decisions about the appropriate cut-off period".

Some overall evaluation comments

For an intelligent virtual advisor on the subject of dissertation project, the assistant bot is able to demonstrate high competence to do so. The advice from the bot is informed, comprehensible and helpful. It serves as an intelligent, helpful and highly accessible companion to housing studies students doing dissertation projects. 

Reference: a collection of blog notes on using ChatGPT.