CIS 6930/4930: Research Methods for Human-centered Computing

Instructor: Eakta Jain
Office hours: Tuesday 2:45-3:45 in E540
Time: Tuesday Period 7 (1:55-2:45pm)
Thursday Period 7-8 (1:55-3:50pm)
Classroom: E 221 CSE Building

Description:

Introduces the fundamental methods and techniques to collect data from humans for building and evaluating technologies, including experimental design, types of variables, types of errors, hypothesis testing, survey design, behavioral and psychophysical methods.

Students will learn the fundamental methods and techniques to conduct research into questions that  involve data collected from humans. The class will involve learning through mini-assignments, projects, and active participation. The class is well-suited for PhD students starting on research work involving experimental design, hypothesis testing, or human-centered research, and Masters students interested in pursuing graduate research. Undergraduate students should have a research project that will require these skills if they intend to take this class (e.g. senior thesis, or an independent research study). They should talk to the instructor before registering.

By the end of this course, students will be able to
-    design a hypothesis
-    design a human subjects experiment
-    collect a variety of data from human subjects
-    analyze data via statistical methods

NEW -- CLASS PROJECT VIDEOS NOW ONLINE -- SCROLL DOWN!

Tentative Outline:

The set of topics and their order could change depending on class interest and time.

Week Topic Tuesday
Thursday
Jan 6 and 8 Introduction : What is research
Festival of Learning
Introduction
Example of a well scribed lecture
Jan 13 and 15
Ethics, Methodology
Assign scribes
Research methods: Observational, Experimental, Correlational
Slides
Project details
Ethics in Human Subjects Research, UF CITI training
Sample IRB
Sample Consent form
  Jan 20 and 22
Scientific Foundations

Project pitch (Hypothesis, How will you test it)
5 mins per group
Submit your IRB and consent form in printed form
Types of data, Internal and external validity
week3_lecture2_scribe1.pdf (Phillip)
week3_lecture2_scribe2.pdf (Naja)
week3_lecture2_scribe3.pdf (Tiffanie)
  Jan 27 and 29 Designing Experiments

Independent and Dependent Variables
Control and Random Variables
week4_lecture1_scribe1.pdf (Marvin)
week4_lecture1_scribe2.pdf (Jessica)
week4_lecture1_scribe3.pdf (Dekita)
Confounds
Designing a task
Recruiting participants
week4_lecture2_scribe1.pdf (France)
week4_lecture2_scribe2.pdf (Alison)
week4_lecture2_scribe3.pdf (Jerone)
  Feb 3 and 5 Designing Experiments
Within subjects and between subjects design
Order effects, Counterbalancing, latin squares
week5_lecture1_scribe1.pdf (Yerika)
week5_lecture1_scribe2.pdf (Chris Wan)
week5_lecture1_scribe3.pdf (Sanethia)
Questionnaire design
week5_lecture2_scribe1.pdf (Kai)
week5_lecture2_scribe2.pdf (Subhankar)
week5_lecture2_scribe3.pdf (Vincent)
Feb 10 and 12
Self-report data
Using scales for reliability, Cronbach's alpha
Data for classroom exercise
R notes
week6_lecture1_scribe1.pdf (Stephanie)
week6_lecture1_scribe2.pdf (Zsolti)
week6_lecture1_scribe3.pdf (Forrest-William)
Psychophysical methods
week6_lecture2_scribe1.pdf (Elizabeth)
week6_lecture2_scribe2.pdf (Andrew)
week6_lecture2_scribe3.pdf (Chris Crawford)

Feb 17 and 19 Project mid-term presentations
(Background reading done, study design ready)
Sample (Good job, Andrew, Stephanie and Zsolti!)


 Feb 24 and 26 Behavioral data: eyetracking, reaction times

Reaction times (slide)
week8_lecture1_scribe1.pdf (Phillip)
week8_lecture1_scribe2.pdf (Naja)
week8_lecture1_scribe3.pdf (Tiffanie)
Eyetracking (class slides)

In-class reading: Methods for comparing scanpaths and saliency maps:
strengths and weaknesses


In-class exercise


week8_lecture2_scribe1.pdf (Marvin)
week8_lecture2_scribe2.pdf (Jessica)
week8_lecture2_scribe3.pdf (Dekita)

Mar 10 and 12
Mar 2-6 is Spring Break: You should have piloted your experiment on your two best friends by this time, and modified it if necessary.
Projects
Time to run your subjects for your project
Time to run your subjects for your project

Mar 17 and 19
Take-home mid-term exam
Psychophysics

Updates: Scribed lectures graded (mean=8.45/10, max=10, min=7)
Take home mid-term exam
Psychophysical methods

Visual Illusions
Online Experiments courtesy Hong Tan (Purdue)
week10_lecture2_scribe1.pdf (France)
week10_lecture2_scribe2.pdf (Alison)
week10_lecture2_scribe3.pdf (Jerone)
Mar 24 and 26 Analyze your data



Descriptive statistics (mean, SD, explore the distribution via box plot, outliers)
week11_lecture1_scribe1.pdf (Yerika)
week11_lecture2_scribe2.pdf (Chris Wan)
week11_lecture2_scribe3.pdf (Sanethia)
T-test, one-way ANOVA
week11_lecture2_scribe1.pdf (Kai)
week11_lecture2a_scribe2.pdf (Subhankar)
week11_lecture2a_scribe3.pdf (Vincent)
Mar 31 and Apr 2
Projects + Bonus topic
Creative Campus lecture: Integrating Biology, Psychology, and Dance
week12_lecture1_scribe1.pdf (Stephanie)
week12_lecture1_scribe2.pdf (Zsolti)
week12_lecture1_scribe3.pdf (Forrest-William)
Project updates
  Apr 7 and 9 Model fitting via linear regression week13_lecture1_scribe1.pdf (Elizabeth)
week13_lecture1_scribe2.pdf (Andrew)
week13_lecture1_scribe3.pdf (Chris Crawford)
week13_lecture2_scribe1.pdf
week13_lecture2_scribe2.pdf
week13_lecture2_scribe3.pdf
Apr 14 and 16 Qualitative methods


Guest lecture: Dr. Gardner-McCune
on Interviews and Observations (how to collect data and analyze it)
Project final presentation
GREAT JOB EVERYONE!
Apr 21
Bonus topic
Writing a Research Paper

Class Projects

Great job everyone!

1. Project: The Effect of Footstep Sound on Walking Pace in Different Virtual Environments
Group Members: Stephanie Carnell, Andrew Robb, and Zsolt Szabo
Video: https://www.youtube.com/watch?v=VT57eUK4_Io
Best project by class vote!!

2. Project: Visual Salience and Emotinal Salience in Illustrations
Group Members: Yerika Jimenez, Elizabeth Matthews, and Sanethia Thomas
Video:  http://youtu.be/Q1NXzLjY7T4
Honorable Mention!!

3. Project: Collaborative Brain-Computer Interface
Group Members: Marvin Andujar, Chris Crawford, France Jackson
Link: https://youtu.be/68NvN25NX54
Honorable Mention!!

4. Project: Investigating Perceived Attractiveness by Manipulating the Golden Ratio
Group Members: Dekita Moon and Alison Nolan
Video: https://youtu.be/YOVvheklxT8

5. Project:Icon Effects and Perception of Functionality
Group Members: Jessica Jones, Naja Mack, Tiffanie Smith
Video: https://www.youtube.com/watch?v=Q1mW57nKdOQ

6. Project: Sound Sample Detection and Numerosity Estimation Using Auditory Display
Group Member: Chris Wan, Kai Zhang, Vincent Ciaramella
Video: https://www.youtube.com/watch?v=6L9qTiuhsJU

7. Project: Visual Equivalence of Mobile Imagery vs Reality
Group Members: William Forrest Nelson and Subhankar Mishra
Video: https://youtu.be/26yLjeiS2q4

8. Project: Does race affect Human Perception of digital photographs
Group Members: Phillip Hall and Jerone Dunbar
Link: http://youtu.be/88TkMmGyED0

Other Information

On any department-managed Unix or Linux machine, your class space directory/
directories is/are:

/cise/class/cis6930sp15res

On any department-managed PC running windows, your class space directory/
directories is/are:

Z:\cise\class\cis6930sp15res

To put the lecture notes up:
ssh into a CISE server, e.g., sand.cise.ufl.edu
name your pdf file week<NUMBER>_lecture<NUMBER>_scribe<NUMBER>.pdf
copy it to public_html
chmod a+r

Lecture notes should be online in one week's time. (We'll check in class!)

Scribing lectures

A few general comments:

1) The scribed lecture notes should describe the subject matter not
the dialogue in the class. You want to cover the material that was
raised as answers to questions/in-class discussion, but not as
dialogue. Please see the example of a well scribed lecture on the
class webpage.

2) Please clearly mark out your additional example.

3) Write your names on the lecture notes.

4) Please complete sentences, run spell-check, and pay attention to
grammar. Writing skills develop through practice.

Resources for R

Get R here: http://lib.stat.cmu.edu/R/CRAN/

Useful notes for simple operations, reading files: http://www.luchsinger-mathematics.ch/Bashir.pdf

Psych package for experimental psychology: http://cran.r-project.org/web/packages/psych/psych.pdf

http://www.rexamples.com/

Introductory data analysis for astrophysists (useful for us too... mathematics is universal ... pun intended):
http://sites.stat.psu.edu/~dhunter/astrostatistics/excerpts/

Extra Reading

Questionnaire design:
http://www.statsoft.com/Textbook/Reliability-and-Item-Analysis
http://home.ubalt.edu/ntsbarsh/stat-data/Surveys.htm

Jim Ferwada's Psychophysics 101 class:
http://dl.acm.org/citation.cfm?id=1401243

Practical Statistics for HCI:
http://depts.washington.edu/aimgroup/proj/ps4hci/

http://www.surveygizmo.com/survey-blog/
what-you-need-to-know-when-using-word-clouds-to-present-your-qualitative-data/