Principles of Scientific Research

Five Key Characteristics

Scientific research in psychology is guided by several foundational principles that ensure the validity and reliability of findings. These characteristics distinguish scientific inquiry from other forms of knowledge acquisition.

• Based on measurements: Research relies on quantifiable data.

• Generalizable: Findings should apply beyond the specific study sample.

• Techniques that reduce bias: Methods are designed to minimize subjective influence.

• Made public: Results are shared for scrutiny and replication.

• Replicable: Other researchers should be able to repeat the study and obtain similar results.

Scientific Measurement

Objectivity in Measurement

Objective measurement is essential for scientific validity. It involves quantifying entities or behaviors in a consistent manner, independent of subjective interpretation.

• Objective measurements: The measure of an entity or behavior that, within an allowed margin of error, is consistent across instruments and observers.

• Example: Measuring weight with a scale; the margin of error is specified in the definition.

• Operational Definitions: Variables are defined by the specific procedures/measures used to record observations.

Technological Advancements in Measurement

Modern technology has expanded the ways psychologists can measure and study psychological phenomena.

• fMRI: Functional magnetic resonance imaging tracks brain activity during tasks (e.g., remembering words, viewing emotional images).

• Physiological measures: Blood or saliva samples analyzed for hormones, etc.

• Multiple measurement methods: Using different approaches for the same variable (e.g., anxiety) improves understanding.

Importance of Definitions in Research

Clear definitions are crucial for reliable and valid research.

• Psychological concepts: Require carefully defined terms (e.g., personality, shyness, cognitive ability).

• Operational definitions: Specify the exact procedures used to measure variables.

• Applications: Planning studies, sharing and comparing results.

Operational Definitions Table

Operational definitions clarify how abstract concepts are measured in research.

Validity and Reliability

Definitions

• Validity: The degree to which an instrument or procedure actually measures what it claims to measure.

• Reliability: The degree to which an instrument provides consistent and stable answers across multiple observations and points in time.

Test-Retest Reliability

• Examines whether scores on a test remain consistent over time.

• If the same individual completes a test at two different times, scores should be similar if the construct is stable.

• Example: A depression questionnaire should yield similar results on different occasions if depression levels remain the same.

Alternate-Forms Reliability

• Evaluates whether different versions of the same test produce equivalent results.

• Useful when repeated testing is needed (e.g., brain injury patients).

Inter-Rater Reliability

• Applies when behavior or responses are scored by human observers or raters.

• High inter-rater reliability is achieved when raters use clear operational definitions and criteria.

• Example: Psychologists might videotape people interacting and have raters score open-ended responses.

Generalizability of Results

Definition and Application

Generalizability refers to the degree to which one set of results can be applied to other situations, individuals, or events.

• Sample: A select group of population members.

• Random sampling: Each individual in the population has an equal chance of being included.

• Convenience samples: Samples of individuals who are most readily available.

• Caution: Results should not automatically be applied to all groups; differences (e.g., age, background) may affect generalizability.

Bias in Research

Types of Bias

• Researcher Bias: When experimenters unintentionally influence results.

• Participant Bias: Demand characteristics; participants guess study purpose and change behavior.

• Observation Effects: Hawthorne Effect—participants change behavior because they are being observed.

• Placebo Effect: Measurable improvement due to expectations, not treatment.

Demand Characteristics and Participant Behavior

• Psychological studies should minimize behavior contamination from bias.

• Solutions: Anonymous responses, clear info about data use, reducing anxiety.

Classic Studies on Demand Characteristics & Bias

• Rosenthal & Jacobson (1966): Teachers told some students had unusual potential—students improved in grades due to teachers' positive expectations.

• Rosenthal & Fode (1963): "Bright" vs. "dull" rats—rats performed differently based on experimenters' expectations.

Controlling Bias

Methods

• Control: Rigorous training and use of scripts; post-study interviews/questionnaires.

• Relevance & Consequences: Bias threatens validity; especially harmful in clinical research.

• Key Takeaway: Demand characteristics and experimenter bias must be minimized.

Techniques That Reduce Bias

• Anonymity: Responses not linked to names/identifying info.

• Confidentiality: Only researchers see results.

• Placebos: Control for placebo effect; participants must believe they might be receiving the real treatment.

• Single-blind study: Participants don't know the study's true purpose or whether they receive drug/placebo.

• Double-blind study: Neither participants nor researchers know who receives which treatment.

Sharing the Results

• After conducting objective, bias-free experiments, findings must be shared.

• Journals: Periodicals with multiple articles; can be technical/specialized or general.

Replication in Research

Definition and Importance

• Replication: Repeating a study to see if similar results occur.

• Ensures findings are not just due to chance.

• Objective results and correct hypotheses should replicate reliably.

Replication Crisis

• Many published studies fail to replicate.

• Example: Open Science Collaboration (OSC, 2015) tried to replicate 100 published studies; only 36-47% replicated successfully.

• Publication bias: Journals prefer publishing positive results; failed/no-effect studies often unpublished.

Challenges & Solutions

• Original study vs. failed replication dilemma.

• Meta-replication projects show many original findings repeatedly fail to replicate.

• Key Takeaway: Replication strengthens psychology by filtering out flukes and methodological errors.

Poor Research & Weak Evidence

Five Characteristics of Poor Research

• Good research is valid, objective, reliable, replicable.

• Poor research lacks these qualities.

Common Signs of Poor Research/Evidence

• Untestable Hypotheses: Claims cannot be tested scientifically.

• Anecdotes/personal experiences: Not systematic or generalizable.

• Biased data selection: Only data supporting the claim is used.

• Appeals to authority: Claiming something is true because an "expert" said it.

• Appeals to common sense/tradition/novelty: Using intuition, tradition, or "latest trend" as proof is unreliable.

Characteristics of Poor Research / Weak Evidence Table

Key Takeaways

• Weak evidence is untestable, anecdotal, selectively chosen, authority-based, or common-sense-based.

• Strong scientific claims rely on falsifiable, systematic, objective, and peer-reviewed evidence.