A brief note on evaluation research for dissertation project works

A brief note on evaluation research for dissertation project works (information from perplexity.ai)

My question 1: "From my teaching of Research Methods for the MBA and Housing Studies students, I notice that quite some of their dissertation projects involve evaluation research, e.g. on business strategy performance and innovation performance [for MBA students] and e.g. housing policy effectiveness and urban renewal impacts [for Housing Studies students]. In this regard, please highlight 6 main ideas of evaluation research methodology and 6 main ideas of evaluation research method."

The reply: "Here are the 6 main ideas of evaluation research methodology and 6 main ideas of evaluation research method based on contemporary understanding:

Main Ideas of Evaluation Research Methodology

1. Systematic Approach: Evaluation research follows a clear, systematic process with defined steps—setting objectives, selecting methods, collecting data, analyzing, and reporting findings—to ensure rigor and consistency.

2. Objectivity and Impartiality: Emphasis on neutrality where researchers avoid bias, ensuring that outcomes are based on evidence and credible for stakeholders.

3. Context-Specific: Tailors the evaluation to the unique environment, population, and contextual factors influencing the program or policy evaluated for relevant and actionable insights.

4. Utilization-Focused: Designed to provide useful information for decision-making and improving programs or policies, engaging stakeholders throughout for practical impact.

5. Scientific-Experimental Orientation: Often draws from social science experimental and quasi-experimental designs that prioritize validity, accuracy, and objectivity.

6. Comprehensive and Management-Oriented Models: May incorporate systems models like CIPP (Context, Input, Process, Product) that evaluate programs within the broader organizational or policy framework.

Main Ideas of Evaluation Research Method

1. Planning and Design: Defining clear evaluation objectives, criteria for success, and choosing appropriate qualitative and/or quantitative methodologies.

2. Data Collection: Gathering data through various methods such as surveys, interviews, observations, focus groups, and existing records to capture relevant information.

3. Data Analysis: Applying statistical techniques to quantitative data and thematic analysis to qualitative data to identify patterns, trends, and outcomes.

4. Needs Assessment and Formative Evaluation: Methods to identify the problem scope, severity, and appropriate program delivery mechanisms.

5. Summative Evaluation Techniques: Use of observational, correlational, quasi-experimental, and experimental designs to assess program effectiveness and impact.

6. Cost-Effectiveness and Impact Assessment: Employing econometric methods and qualitative evaluation to measure the net impact, cost-benefit, and overall value of the program or initiative.

These points synthesize best practices for both the overarching methodology framework and the methods specifically used for conducting evaluation research in business strategy, innovation, housing policy, and urban renewal contexts relevant to MBA and Housing Studies students."

My question 2: "Suggest specifically 2 academic approaches and 4 academic theories for this topic and provide a few references of them in Harvard reference format."

The reply: "For evaluation research in the context of MBA and Housing Studies dissertation topics, here are 2 academic approaches and 4 academic theories that are highly relevant:

Academic Approaches

1. Theory-Based Evaluation Approach: Focuses on examining the theoretical framework or "program theory" behind an intervention or policy to understand the causal mechanisms and context influencing outcomes. This helps in explaining how and why a program achieves its impacts. (Reference: Weiss, 1997; Pawson & Tilley, 1997)

2. Utilization-Focused Evaluation: Emphasizes evaluation processes and results designed specifically for practical use by intended users and stakeholders, making the evaluation more relevant and actionable. (Reference: Patton, 2008)

Academic Theories

1. Realist Evaluation Theory: Assumes that programs work differently depending on context because mechanisms interact with various social and environmental factors to produce outcomes. (Pawson & Tilley, 1997)

2. Logic Model/Theory of Change: A visual or narrative model linking program inputs, activities, outputs, and expected outcomes that guides evaluation design and clarifies assumptions. (Rogers, 2008)

3. Systems Theory: Views programs as complex systems with interrelated components; useful to evaluate dynamic interactions and emergent outcomes in complex social settings like urban renewal or housing policy. (Checkland & Scholes, 1990)

4. CIPP Model (Context, Input, Process, Product): An evaluation framework that assesses contextual needs, program inputs, implementation processes, and outcomes/products holistically. (Stufflebeam, 2003)

References in Harvard format

• Patton, M.Q., 2008. Utilization-Focused Evaluation. 4th ed. Thousand Oaks, CA: Sage Publications.

• Pawson, R. and Tilley, N., 1997. Realistic Evaluation. London: Sage Publications.

• Rogers, P.J., 2008. Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions. Evaluation, 14(1), pp.29-48.

• Stufflebeam, D.L., 2003. The CIPP Model for Evaluation. In: T. Kellaghan and D.L. Stufflebeam, eds. International Handbook of Educational Evaluation. Dordrecht: Springer, pp.31-62.

• Weiss, C.H., 1997. Theory-Based Evaluation: Past, Present, and Future. New Directions for Evaluation, 1997(76), pp.41-55.

• Checkland, P. and Scholes, J., 1990. Soft Systems Methodology in Action. Chichester: Wiley.

These approaches and theories will strengthen the conceptual and methodological basis of evaluation research projects on business strategy, innovation, housing policy effectiveness, and urban renewal impacts relevant to MBA and Housing Studies students."

My question 3: "Are there any evaluation methodologies or methods that are more based on interpretivism, social constructionism, or critical realism?"

The reply: "Yes, there are evaluation methodologies and methods that are based on interpretivism, social constructionism, and critical realism. These paradigms emphasize understanding social reality as constructed through human experiences, meanings, and contexts, rather than assuming an objective independent reality.

Interpretivism and Social Constructionism-Based Evaluation Methodologies

• Interpretive Evaluation: Focuses on understanding programs or interventions by interpreting the meanings and experiences of participants in their natural social and historic contexts. Methods typically include qualitative approaches like in-depth case studies, ethnography, participant observation, and interviews. The process is iterative, involving reflexivity and searching for "thick description" to capture participants' perspectives authentically. This approach values credibility, dependability, and transferability over positivist notions of validity. (Walsham, 1993; Lincoln & Guba, 1985)

• Contextual and Participative Evaluation: These methods involve stakeholders in the evaluation process, fostering co-construction of knowledge and shared meanings about what interventions achieve and how. Participative methods focus on empowerment and practical usefulness of evaluation, often employing ethnographic and action research techniques. (Susman & Evered, 1978)

Critical Realism-Based Evaluation Methodologies

• Realist Evaluation: Rooted in critical realism, this methodology seeks to uncover underlying generative mechanisms that produce outcomes in particular contexts. It recognizes that social programs work differently depending on the contextual interplay of mechanisms and conditions, going beyond surface-level observations to explain causal pathways. (Pawson & Tilley, 1997)

• Critical Interpretive Research: Combines interpretive approaches with a critical lens that questions power structures and aims to reveal social inequalities embedded in program processes and outcomes. Researchers engage in reflexivity and critical interpretation to challenge dominant narratives and promote social change. (Golden-Biddle & Locke, 1997)

These approaches and methods provide frameworks for evaluation that are less about objective measurement and more about understanding social complexity, meaning-making, and causal explanation within real-world contexts."

A collection of blog notes on using chatgpt for research purpose.