And actually, unlike last year, I won’t be writing my textbook anymore because it’s been published! If you would like a student-centered, practical, one-semester introduction to statistics for business, I’d (obviously) strongly recommend my Step-by-Step Introduction to Statistics for Business.  Each chapter starts with a case study describing a small business owner facing a problem that can be solved with statistics.  Lots of diagrams and illustrations too!  If my post on intra-class correlations is any indicator, apparently I’ve got a knack for describing statistics pretty clearly.

So instead of the textbook, I’ll instead be working on journal article submissions, a couple of software design projects, and taking a special trip to Europe for my 5-year wedding anniversary.  A busy summer!

Problematic online gaming is related to low self-esteem, depression, and other negative consequences to your psychological health. Below, you can complete the Problematic Online Gaming Questionnaire (short-form) and get an assessment of your own addiction. If you wish to assess someone else, it is important that they complete the assessment themselves; don’t read it to them. After you submit your answers, additional analysis of your responses will appear comparing you to the sample collected by the research team and indicating your risk factors.

Please note: You must respond to every question to learn your results. Your responses are entirely local to your own computer (your responses will remain unknown to anyone but you).

1. Pápay, O., Urbán, R., Griffiths, M., Nagygyörgy, K., Farkas, J., Kökönyei, G., Felvinczi, K., Oláh, A., Elekes, Z., & Demetrovics, Z. (2013). Psychometric Properties of the Problematic Online Gaming Questionnaire Short-Form and Prevalence of Problematic Online Gaming in a National Sample of Adolescents Cyberpsychology, Behavior, and Social Networking DOI: 10.1089/cyber.2012.0484
2. Demetrovics, Z., Urbán, R., Nagygyörgy, K., Farkas, J., Griffiths, M., Pápay, O., Kökönyei, G., Felvinczi, K., & Oláh, A. (2012). The Development of the Problematic Online Gaming Questionnaire (POGQ) PLoS ONE, 7 (5) DOI: 10.1371/journal.pone.0036417

The state of [industrial/organizational psychology] and its obsession with novel theoretical contributions is antithetical to the goals of the scientific method.

The authors lament the over-reliance on the development of novel theory as the foundation for a meaningful contribution to scientific research. They suggest, rightfully I think, that the pursuit of novel theory creates a perverse incentive structure for scientists, encouraging them to pursue identification of “facts” which may fit available data but do not really exist, ultimately leading to the creation of theories that do not truly complement or build upon any prior theories, orphans of the scientific literature that provide no true value to our understanding of organizational phenomena.  They label this approach the pursuit of the “interesting” at the expense of good science.

They further argue that this state of affairs is in fact the result of a fundamental misunderstanding of the scientific process.  The enterprise of science is not one where we should aim to identify facts, because facts do not really exist; instead, it is one where theory is used as a tool to incrementally better understand phenomena, each additional bit of research contributing to a better description of a real research problem.  When we pursue conclusions like “this is how organizations function” or “this is how employees behave”, we are misleading both ourselves those that rely upon our research.

Certainly, such misunderstandings have occurred before, with highly damaging implications for the progress of science. One such misunderstanding was the over-reliance on statistical significance testing, where “statistically significant” was mistakenly taken as synonymous for “real” or “important”.  Although people still make such mistakes in interpretation, the movement toward effect size interpretation and model specification (as evidenced through changing APA guidelines for reporting) is clearly targeted at reducing this continuing problem.  In their article, Pillutla and Thau are arguing that theory development as the sole target of science is the research-methods equivalent of the statistical-significance interpretation problem – that this over-emphasis in academic publishing is pervasive and harmful.  Like statistical significance testing, sometimes novel theory development is the right approach, but many times it isn’t.

Through analysis of citations to one of the most dominant proponents of theory-building as the basis of all science, the authors also contend that management, business, and applied psychology have been disproportionately harmed by this viewpoint, in comparison to other areas of the social sciences.

This view is a bit extreme, but may be the front edge of a coming wave of reform.  I’ve noticed myself over the last few years that fewer and fewer papers appearing in our top journals - Journal of Applied Psychology, Personnel Psychology, Academy of Management Journal, etc. – have solved any real problems for actual organizations.  Instead, they focus on exploring and specifying increasingly minute aspects of organizations – interesting, sure – but not terribly useful in any tangible sense.  From chatting with practitioners and other early career scholars, I’ve discovered this view is not all that unusual.  One person at the Society for Industrial and Organizational Psychology conference this year even told me, “I haven’t seen anything I could actually use to help my employees in Journal of Applied Psychology for at least five years.”  That is a terrible state of affairs.  If we’re not solving real problems, what value do we really provide to both organizations (whom we study) and taxpayers (who pay us, or at least those of us working in state institutions)?  The push to publish only in top tier journals – where such theory development is required – only exacerbates this problem.

Interesting theories are not, by themselves, worthy of investigation unless they are put to the service of explaining research problems… the quest for novelty and interestingness of facts has infused them with significance without any regard to the knowledge that they generate.

So are things as dire as Pillutla and Thau indicate?  There have certainly been a number of high-profile cases in psychology lately that have caused the public to question the value of our entire field.  And we should absolutely work to repair that damage, but the path to do so is unclear.  Some researchers have taken a purely empirical approach, attempting to replicate controversial papers and examine their convergence, eschewing theory entirely in the quest for reliable knowledge.  Is that enough?  I’m guessing we’ll find out in the next five or ten years.

1. Pillutla, M., & Thau, S. (2013). Organizational sciences’ obsession with “that’s interesting!”: Consequences and an alternative Organizational Psychology Review, 3 (2), 187-194 DOI: 10.1177/2041386613479963

Day 3 was the final day of the SIOP conference. Per tradition, sessions were lightly attended – too much late night enjoyment at the end of Day 2, I suspect!  I had a slow start myself, only getting to poster sessions a little after 10AM.  One that caught my eye explored the effect of organizational support for the use of technology over time; surprisingly, the effect of strong organizational support on technology use weakened over time, not quite in line with what prior theory (and the researcher presenting!) predicted.

I also decided somewhat last-minute to attend a Master Tutorial on Bayesian statistics for I/O.  Unfortunately, despite a full house and CE credits on the line for many, the speaker didn’t show!  After about 15 minutes, I left and headed back to the poster session on Research Methods, with lots of interesting work, including a bit more on the equivalence of mTurk and undergraduate research samples.  The added value on this paper?  English-speaking, US-based mTurk samples are similar to undergraduate samples, but non-English-speaker samples are a little different.

Finally, I ended up at the closing plenary, given by Reverend TJ Martinez, founder of the Cristo Rey Jesuit College.  He related an inspiring story about starting with zero support to start a school but ending by graduating a fair number of low-SES high school students (with an abysmal high school completion rate prior) with a 100% entrance rate into college. I am still not sure precisely what he had to do with I/O, but he was certainly an entertaining and admirable speaker.

At the very end incoming SIOP President Tammy Allen laid out a 5-section plan for SIOP in the coming year:

1. Tie the varied local I/O advocacy and interest groups better to SIOP.
2. Increase the visibility of I/O Psychology in Introductory Psychology courses and textbooks.  Graduates of undergraduate psychology programs should, at minimum, be able to answer the question, “What is I/O Psychology?”, and this doesn’t happen in most programs.
3. Work to improve SIOP’s branding, which may include the launch of a new practitioner-oriented journal.
4. Improve national advocacy efforts on behalf of SIOP.  Apparently we have already some folks on retainer, but this will expand.
5. Better understand SIOP’s role and position in “science.”  Psychology is viewed as one of the foundational areas of scholarship on which many other areas are based, but where does I/O fit specifically?

And that’s it for SIOP 2013!  I’ve got another day in Houston – which will likely involve the Houston Aquarium and a nice dinner somewhere – and then back to cooler Norfolk (at a relatively chillier 70 degrees F!), where I’ll be writing a slew of follow-up emails to all the intriguing folks I met this year.  And hopefully I’ll be sufficiently recovered from this trip in a year to attend SIOP 2014 in sunny Honolulu!

Day 2 of SIOP 2013 was a bit busier for me. After an early breakfast, I headed to my first session of the day at 8:30, a discussion of social media in selection, which ended up really being a discussion of LinkedIn in selection. Like Day 1, this was a bit depressing. Although there was a lot of talk about social media, there was not much data, which is what I was really hoping for.  I did hear a few interesting tidbits in this session that I had not heard before. For example, a lot of managers look up job candidates on LinkedIn, where many pieces of information are shared that would not be part of a job application, like profile pictures and affiliated religious groups.  Once a hiring manager sees such information, their judgment has potentially been irreparably affected – or more critically, the possibility that it has been so affected creates a substantial legal risk. How can an organization prove that manager wasn’t influenced by the job candidate’s race, skin color, national origin, gender or religion once they have been exposed to it?

The next session on social media was a little better, data-wise, but it wasn’t perfect. One of the presentations provided some data on the criterion-related validity of LinkedIn profiles for predicting job performance, as an extension of a previously published paper doing the same for Facebook profiles. As it turns out, LinkedIn profiles can provide valuable information to hiring managers.  But I asked a question of the speakers: Aren’t we just setting ourselves up to redo this same study for every social media platform of the moment? If we seek a scientific understanding of the useful information provided by social media, don’t we need a better way to do it? Everyone seemed to think that was a great idea, but alas, I didn’t get any specific recommendations on how to go about it.

Next, I headed to a symposium on online simulation, which was absolutely packed. Much in line with what I noticed from Day 1, practitioners are hungry for data on how these technologies have and should affect their practice – and yet again, data was a bit light. They presented some interesting assessment tools, but they seemed like relatively small advancements over what we already have – for example, the use of branching video-based simulations rather than branching text or static video assessments.  The practitioners did bring up some interesting issues they faced in launching these technologies globally.  For example, when they tried to launch their video-based simulation in another country with poor Internet infrastructure, they discovered that bandwidth was not the guarantee that it is in the US – their client couldn’t view the videos at even quasi-decent quality. This resulted in some rather last minute changes to meet client needs.

After this symposium, I headed over to the poster session on training – in an hour, I got through only about 75% of the room.  Hopefully I’ll have a bunch of papers incoming to my inbox in the next few days so that I can revisit them in more detail than a 3′ x 5′ poster can provide!

Finally, at 5PM, in a symposium with my colleagues at ICF and ARI on emerging training technologies, I gave my presentation on gamification, which was by a large margin the most successful presentation I have ever given at SIOP in the 9 years I have attended.  In it, I presented a theoretical model of gamification providing two specific mechanisms by which gamification can affect learning in the context of education/training, and I tested one of those processes empirically, providing several practical recommendations for those wishing to gamify their own organizational learning. I had literally 25 minutes of questions and comments after the session ended!

All in all, an exceptionally successful Day 2! Day 3 is a bit lighter, and I suspect tonight will be a bit more intense, so Day 3 may also start a bit late. But I am still hopeful for some great learning and some great conversations.

13:13	Hangovers and spotty session attendance… so starts Day 2 of #siop13… Although many of the hangovers won’t start for hours yet!
13:41	Yet another session on social media for selection… Some data in this one, I hope? #siop13
14:05	Information on protected classes may be more common on LinkedIn than in resumes… Greater legal risk (managers cannot unsee!) #siop13
14:06	@ChrisWieseIO The speakers seem to be mixing selection predictors and selection methods… Seems a very jumbled approach to me. #siop13
14:10	I’m getting a little tired of hearing “maybe we’ll have some empirical research later” in these social media symposia #siop13
14:26	@ericknud I’ve got several running right now… Just sad there are not more here #siop13
14:34	I am sad when people count the number of stat. significant relationships in their results as an indicator of result stability #siop13
14:47	@ericknud Open for now! Stick around, will probably head up front
15:17	@ericknud was talking to Don until just now, out in the hall now for a bagel
15:17	Discovered I was hoarse from last night’s activities while asking a question at a symposium… whoops #siop13
17:12	Full house, standing room only at innovations in online simulation #siop13
17:29	To practitioners, video-based virtual assessment center apparently = long-form video SJTs with open ended responses #siop13
17:36	Cultural issues within orgs can be unexpected and tricky for innovative assessment: “our org doesn’t use email” #siop13
17:55	Online assessment center design with branching reminds me of game design process #siop13
19:02	Did not even get all the way through the training poster session in an hour – too much interesting stuff! #siop13
19:03	RT @sandyfias: It makes me sad that about 8/3800 I/O psychologists at #siop13 r tweeting. And we ask how we can be more visible and rele …
19:03	@sandyfias I’d suggest we have a tweetup, but it would just be depressing! #siop13
20:22	Hope you’re attending my session at #siop13 on serious games and gamification in Room 339AB at 5PM today! (Room change!)
20:25	@BelindaK04 @PsychoSoAnt @sandyfias Are there enough people? I’ve got 3-4:30 tomorrow #siop13
21:55	Starting in 5 mins, session at #siop13 on serious games and gamification in Room 339AB!
22:05	Standing room only at the session! #siop13 http://t.co/HLZwNOwTpY
22:37	RT @jsnread: Gamified learning results in longer time spent on task, thus greater learning. Wish I could have gamified my dissertation. …
22:38	@neilmorelli thanks – got a little rushed at the end but hopefully it made sense! #siop13
23:27	Wow – 25 minutes of questions about #gamification research after session! Thanks everyone! #siop13
23:27	@jsnread ha, why not!?
23:50	#siop13 Day 2 is over! Now to the ODU alumni dinner (where I entertain) and the Minnesota alumni event (where I am entertaining)

SIOP 2013 has begun, and Day 1 was a fascinating set of presentations.  The day starting with the opening plenary by SIOP President Doug Reynolds, talking about Big Data and I/O Psychology’s role in it.  For some reason, we are not leading the way, despite our expertise being in synthesizing, analyzing and interpreting massive datasets about behavior and personal characteristics.  Sounds like it would be a happy marriage to me!

The biggest difference I noticed today in comparison to last year was the massive number of presentations on social media and mobile assessment.  Mobile assessment presentations seem to focus on the apparent equivalence of computer-based and mobile-based assessment, but I’ve been seeing some results contradictory to that… so more research is clearly needed.  Social media research is progressing, but I heard “we don’t have data” way more times than I can even count.  If you don’t have data, why are you presenting at SIOP?  This is a data-hungry crowd!  Perhaps the most disappointing was the session on “Empirical Evidence” for social media, but where only one presentation had any real data to speak of.  I was hopeful though.

Day 1 was actually my lightest day, and I had a couple of time slots that were not scheduled.  No worries – just walk 10 feet at SIOP and you’ll run into someone you know.  The disadvantage is that it takes an hour to get from one side of the conference hotel to the other!

I also sat in a presentation where “Millenials” were predicted to be dramatically different from everyone else.  Surprise!  I hate such conclusions – Millenials are a diverse bunch, like every other generation.  And I don’t say that just because I’m a Millenial.

Evening activities went pretty late into the night… but it’s SIOP… so that’s just what happens.  Totally worth it.  Day 2 summary  tomorrow!

Archive of Day 1 Twitter posts follows:

As in 2010 and 2011 and 2012, I’ll be live-blogging the SIOP conference, which begins Thursday, April 11 and runs through Saturday, April 13.  This post contains a list of all the sessions that I interested in attending, which are generally focused on technology, training, and assessment.  Barring any unexpected battery problems, my live blogging will be on Twitter, with a permanent record stored here.

In the graphic below, the symposium that I presenting in is colored red.  As you can see, it’s a bit packed, so I won’t be attending all of these, but this is everything I’m interested in.  If you’d like to meet up at any of these events, or if you think I missed something that I should definitely attend, please let me know!

In a recent issue of Cyberpsychology, Behavior and Social Newtorking, Peng and Crouse^[1] compared enjoyment, future play motivation and physical intensity on XBOX 360 game Kinect Adventures between single player, cooperative multiplayer and competitive multiplayer modes, finding that competitive multiplayer provided the greatest enjoyment, motivation to play in the future, and physical activity.

The highly physical core game mechanic is best described by the authors:

This game utilized the Kinect motion camera, and the players used their own bodies as the controller to pop soap bubbles that floated between holes on the walls, floors, and ceilings of a virtual zero-gravity room by moving forward, backward, left, and right and waving their arms.

These three conditions were more complex than they appear:

1. In the single player condition, participants played alone, twice.  In this way, they were competing with their pre-test score.
2. In the cooperative mutliplayer condition, participants brought a friend.  After individual pre-tests, the two participants next played the game with each other (they were physically active in the same space simultaneously and their scores were combined).
3. In the competitive multiplayer condition, participants brought a friend.  After individual pre-tests, the two participants then played the game separately (in different rooms) and were told what their friend’s previous high score was on the pre-test.

After the game, participants completed the focal measures.  Enjoyment was assessed with a 7-item scale asking them to rate the game on seven adjectives (including “boring,” “entertaining,” and “interesting”).  Future play intention was rated on a 3-item scale asking questions like, “Given the chance, I would play this game in my free time.”  Physical activity was captured with an accelerometer attached to each player.

162 students participated in the study; however, this seems to include the friend counts.  Sample sizes were 26 for single player, 74 for cooperative play and 52 for competitive play.  Given that their analytic strategy was a simple ANOVA, it does not seem that the researchers appropriately controlled for covariance between friend pairs – a better strategy would have been to have not included the “friend” in analysis, however this would have dropped their apparent (although misleading) reported sample size.

Based upon the reported ANOVA, several were statistically significant.  Single player enjoyment was lower than cooperative and competitive multiplayer modes.  Future play motivation after playing alone was lower than both cooperative and competitive mutliplayer modes.  For physical activity, the relationship was more complicated.  In this case, the single player and cooperative mutliplayer modes resulted in greater physical activity than the competitive multiplayer.

The authors did not depict these relationship graphically (always graph your results!), so I did:

Created from Table 1 of Peng & Crouse (2013)

Although the affects appear pretty substantial in terms of effect size (for example, from 3.99 to 5.22 for enjoyment, about 0.8 standard deviations), it’s hard to say how much the double-sample for the two multiplayer modes inflated the F-statistics (and thus statistical significance) of the results.  So I am not 100% confident in the researchers’ interpretation of these data, but I am cautiously optimistic that these results would hold up with appropriate consideration of dependence.

In the context of education and training, this has some critical implications: play with others seems preferable to play alone.  While this might seem intuitive, we too often build serious games and gamification efforts where people compete against themselves (e.g. high scores, or some instructor-set target goal level).  Adding some coworkers or fellow students to the mix might just improve those efforts.

1. Peng, W., & Crouse, J. (2013). Playing in parallel: The effects of multiplayer modes in active video game on motivation and physical exertion Cyberpsychology, Behavior, and Social Networking DOI: 10.1089/cyber.2012.0384

In an upcoming issue of Academy of Management Perspectives, Mainiero and Jones^[1] explore the various research literatures related to textual harassment.  It is a complex problem with many grey areas and ambiguities, as  illustrated in this quote:

If a coworker admires an employee’s new dress while at work, that coworker can read tone and body language and politely say thanks.  However, a late night text message from a colleague’s personal account about a new dress takes on a much more lascivious tone, even if that is not the intention.

In their article, the authors focus on textual harassment within the context of workplace relationships, where matters become even more murky.  The authors explore the issues from three perspectives:

1. Workplace surveys reveal that workplace romances are very common.  In one study, more than 33% employees reported finding love in the workplace, and 66% of those employees revealed their relationship to coworkers and/or bosses.  However, 6% of employees revealed that they had left a job due to such a romance.  The biggest challenge for HR in workplace romances is determining if the relationship is ethical (e.g. one employee should not be in a position of power over another, or be able to show favoritism in decision-making).  38% of employees reported that they believed coworkers to gain advantages due to relationships and 31% of employees were uncomfortable with this.  Social media is a relatively common way for such romances to take place, especially Millennials.
2. Sexual harassment after a workplace romance is more common.  And social media makes HR’s responsibilities unclear when such relationship go bad.  Consider this vignette shared by the authors:
   

   Consider the following social media contemporary romance scenario: a woman meets a man at work in her same department. She texts him for a drink after work; he complies. They embark on a romance that lasts three months. He complains that she expects him to do her work for her. She complains that he is too clingy. They part ways, initially amicably, but he continues to text her at work during the day with sexting-related comments about her legs, how her clothing moves as she walks down the hallway, and other intimate details of their past relationship. But in business meetings he is all business and so is she. Outside the office, she blocks him from her Facebook page, yet he still has memorized her cell phone number and continues to bother her during the day and evening hours. He checks her whereabouts on Foursquare. She mentions to him that she does not want him to continue to IM her or follow her on Twitter; he refuses to comply with her request. At work, they are assigned a similar departmental project that requires frequent meetings. They retain a LinkedIn association as colleagues.

   At present, this remains a murky grey area, although several major textual harassment cases are pending.

3. Sexual harassment policies are outdated given this new technology.  Social media, and its dramatic effect on interpersonal behavior at work, has triggered a sudden re-examination of sexual harassment policies.  Such policies are usually driven by state-mandated requirements, but these requirements are often themselves formed as a reaction to legal challenges and public opinion, which are not currently very stable.  Right now, there are no right answers.

The authors conclude from these perspectives that right now, corporations have a responsibility to say “harassment is harassment”, whether it is public or private, and whether it involves personal or corporate resources.  This is certainly the safest route.  However, the authors also recognize that there are threats to privacy inherent to this view.  They also note that workplace romances can be quite positive for everyone involved (the romantic partners and the organization), so prohibiting such relationships is not a reasonable solution.

They suggest implementation of the fabulously labeled love contract, an agreement asking employees entering into a romance to formally report that they are doing so voluntarily (this reminds me a bit of Futurama’s Form B: Notice of Romantic Entanglement, which is filed – where else – at the Central Bureaucracy).

Finally, they suggest that firms implement two types of policy: 1) a proactive policy including discussion of all of these issues in sexual harassment training and 2) a reactive policy explaining the precise procedures for resolving such situations when social media is involved.

While this certainly won’t solve all the problems introduced by new technologies, the authors provide a reasonable first step toward clarifying the role of HR in this brave, new world of social media and romance.

1. Mainiero, L., & Jones, K. (2013). Sexual Harassment Versus Workplace Romance: Social Media Spillover and Textual Harassment in the Workplace Academy of Management Perspectives DOI: 10.5465/amp.2012.0031

I think it’s important to recognize that online doctoral programs as they exist now and online doctoral programs as they could exist, even given current technology, are very different. We currently have the technology available to have the traditionally rigorous, intensive doctoral experience for most students – I just don’t know of any programs that actually do so now. There are several online Psy.D. programs, but requirements for such programs are not as intense, as the degree is practice-oriented – the requirements for a Ph.D. program are more stringent.

So as a thought experiment, I decided to map  traditional Ph.D. student experiences onto the technologies and program structure  that would be required to realize that program online effectively.  In doing so, I came up with five dimensions – the three traditional components of academic performance (teaching, research, and service) plus an instructional component and a community component. I’ll discuss each in turn.

1. Teaching Experience.
   1. Challenge. In a traditional Ph.D. program, students often (although not always) gain teaching experience by being assigned as teaching assistants (TAs), lab/section leaders, and full-blown instructors.  Many schools use a ramp-up program such that earlier students (1st and 2nd year, for example) are TAs or lab leaders, and after proving themselves in that role (or after getting their MA/MS), move up to teach their own courses. If students wish to gain experience in online courses, they sometimes have that freedom; in an in-person program, one can teach in-person or online courses.  In an online program, students can still certainly work as TAs or instructors of online courses, but since there are no in-person classes, they cannot gain experience teaching one.
   2. Roadmap. The dirty secret that no one wants to admit to is that, to a certain degree, teaching is teaching. If you look at programs designed to assess the quality of online courses, you’ll find that what makes a high quality course online is very similar to what makes a high quality course in person.  For example, Quality Matters is a “certification” program designed to assess such quality.  Here are the dimensions it uses to make evaluations: inclusion of a course overview and introduction, statement of learning objectives, appropriate assessment strategies, informative instructional materials, tools to encourage interaction and engagement with and between students, appropriate use of technology, support for learners, and accessibility.  On its own, this list could apply to either an in-person or an online course.  The reason that online courses are often lower quality is that, for the instructor, it is often more difficult to realize these goals online.  In an in-person course, I can split people into small groups to discuss a difficult question; online, I need to adequately plan ahead to set up a discussion area, ensure it’s communicated to everyone appropriately, identify any technology roadblocks and proactively work to circumvent them, and then monitor the discussion over a long period of time.  If they are different, teaching online is more difficult than teaching in person.  It requires most of the same skills and then several additional technology skills.  Where online teaching is less difficult for the instructor is the face-to-face interaction component.  You don’t need to be “on”, excited and engaged at 8AM.  You don’t need to learn to watch and interpret subtle facial cues that indicate students don’t quite grasp the concept you are discussing.  But this is a relatively minor aspect of teaching, in the grand scheme of required skills.
   3. Ideal Solution.  Given that teaching online is more difficult than teaching in-person except in terms of interpersonal interaction, the ideal online Ph.D. program would have the Ph.D. student start as an online TA and move to online instruction.  After “proving” themselves online, the Ph.D. student should teach as an adjunct professor at local universities or community colleges, and the online university should cover the host university’s salary requirements. For example, if the student’s local institution pays $3000 for an adjunct to teach a class, the online university should pay $3000 to the student to teach on behalf of the host institution.
2. Research Experience
   1. Challenge.  In a traditional Ph.D program, students gain research experience by actively designing, running, analyzing, interpreting, and writing up research studies. Most of these experiences involve working closely with an academic adviser who creates these studies and runs them. Such studies are often run by Ph.D. student in in-person laboratories, which require direct oversight of research participants.
   2. Roadmap. In truth, a lot of research in psychology these days is conducted through online surveys. This would be ideal for an online Ph.D. student, requiring few special accommodations. Much research (my own included) incorporates experimental designs but does so online – for example, assigning different research participants to different stimulus materials and measuring outcomes through a follow-up survey. For research that is in-person and experimental in nature, requiring fine timings or specialized equipment, there is currently no easy (or more critically, valid) way to conduct such studies online.
   3. Ideal Solution. For Ph.D. programs that absolutely require in-person experience or equipment (e.g. if an fMRI must be used, or if clinical populations must be interacted with, etc.), such programs should not have online programs without partnerships with universities local to the student.  Ideally, a network of such inter-relationships between universities could be established to facilitate this, but I don’t see this happening any time soon.  If a program relies primarily on survey research, Skype or other videoconferencing software can be used to interact with the adviser and with the various necessary committees (e.g. dissertation committees).  Research experiences must be integrated as of the first year of study.  I can’t even count the number of times I’ve seen students in online Ph.D. programs asking for advice on LinkedIn or on Listservs on how to design studies or conduct analyses for their dissertations because it was the first time they ever had to conduct a research study in their entire graduate program.  This is not acceptable.  A Ph.D. program must incorporate research from Day 1.  Perhaps most critically, universities must keep the student-to-faculty-adviser ratios similar to that of in-person classes (for Ph.D. students, I’d characterize 8 advisees per faculty member as a large program; ideally, this would be in the 3-6 area).  The workload is no lower for faculty, and for quality standards to remain high, a high degree of individual attention is critical.
3. Service Experience
   1. Challenge. Graduate school often brings with it a myriad of service opportunities. In my graduate school experience, this included things like hosting students for welcome/interview weekend, running a student group, organizing brownbags, and so on. This communicates the shared mental model of academia to students: everyone pitches in a little bit to get things done, and so that everyone’s voice is heard.
   2. Roadmap. In an online environment, there is no reason that service opportunities would be lessened. For example, there should be an online student association that organizes online brownbags through Skype or Google Hangouts. A leading scholar could give a talk, with individual live cameras on every other student, all organized by a team of online students. Students could take turns manning the live video “Q&A” area during the official day set aside for interviews of upcoming graduate students. The opportunities to pitch in are endless for a motivated student.
   3. Ideal Solution. Of the five dimensions I’ve listed here, service is probably the easiest. It only requires a faculty willing to be creative and open-minded about the activities that students take part in. At a minimum, students should run their own graduate student association and hold regular online events.
4. Instruction/Learning
   1. Challenge. One of the most difficult aspects of teaching graduate students is the actual act of teaching. It is generally not effective to teach Ph.D. students the same way you teach undergraduates. With undergraduates, there’s a certain amount of memorization required (e.g. who were the important figures in the development of psychology?), with deeper understanding lightly layered on top of that memorization. If students came out of my I/O Psychology class able to tell me the major functions of I/O, the important controversies surrounding those functions, and be able to apply some of the principles to their own workplaces, I would be thrilled.  At the Ph.D. level, the requirements are much more intense. Not only must Ph.D. students be able to tell you about the controversies, but they must also be able to propose research studies to solve the problems that those principles introduce.  They must be able to read and critically evaluate a research literature for gaps and limitations. These are skills that are not transferred easily in lecture, which is why most Ph.D. programs make heavy use of seminars (for content courses) and labs (for statistics courses).
   2. Roadmap. Recently, my department floated the idea of taking online Master’s students without disrupting pre-existing graduate courses. The biggest challenge in this idea is the integration – having in-person and online students in classes concurrently. But I am confident it is possible. For example, a computer could be set up with Google Hangouts (so that each attending online student would be visible with a webcam on that computer), and that computer could be wheeled into each class where online students are needed to participate.  These systems are actually quite easy to use; when someone starts talking, the focal video window switches to that person so that you can see them full-screen. To the instructor, a window would appear with a person’s face whenever they wanted to ask a question; it would only require turning their head and talking to the camera to talk to that student. Intense, interpersonally-oriented seminars are possible and even simple once set up given current video technology.  This is no longer a limitation of online.  If the in-person component is not required, it gets even easier; instead of rolling a computer in, everyone in the class (professor included) can see each other face-to-face with video streaming.
   3. Ideal Solution. The university’s IT group should invest in a high-reliability high-quality small group video streaming technology. One example is Cisco Jabber. For blended classrooms (with both in-person and online students), a mobile computer should be set up that can be transported from classroom to classroom as needed. Real-time instant support must be provided for the technology to both faculty and students.  Class sizes should be kept under 10 for most topics; exceptions might include some “core” courses, like various statistics courses, research fundamentals, and proposal-writing courses.
5. Community Participation
   1. Challenge. Perhaps the most subtle indirect benefit to attending an in-person institution is immersion within the academic community. When you walk to class, you run into professors who may stop you in the hall to chat. When you conduct research, you interact with faculty to get your IRB approvals, to work within the departmental subject pool, and attend research talks. When you conduct service, you are working directly to the benefit of other graduate students and faculty within your department, college, or university. All of these activities teach you to be part of a larger scholarly community. In many online programs, this sense of community is missing.
   2. Roadmap. To this point, psychology faculty have been lucky in terms of community. By bringing in a graduate student, that student will be exposed to the local scholarly community by default. It requires little or no extra work on part of the faculty. Online, this is not true. Community must be actively built. Videoconferencing is perhaps the most cost effective way to do this. For example, lab meetings could be held weekly or biweekly through videoconferencing to encourage people to make such connections. But this alone is probably not enough to provide a real sense of community.
   3. Ideal Solution. To build community, I recommend a multi-pronged approach. First, videoconferencing should be used within lab groups to encourage small-scale community. Second, a student association with regular meetings should be established and run by the students themselves. Third, regular online brownbags should be established with a technology enabling both participant-to-participant and participant-to-presenter interaction. Fourth, students should be required to attend one conference per year (probably APA), where a hospitality suite would be rented by the online institution to enable everyone to meet face-to-face at least once per year. Preferably, these events would be broken down further to give ample opportunities for labs to meet together as well. Fifth, a regular retreat should be held once per year at a physical location outside of the context of a conference for two to three days, to enable long-term planning among labs and conduct teambuilding exercises more broadly.

So as you can hopefully see, what I would consider a high-quality rigorous online psychology Ph.D. program would require quite a great deal of effort on the part of a department chair and likely several committees. It is not something that can be started lightly, or else your program will end up with the reputation of current online Psychology programs, which is frankly pretty terrible.  And that doesn’t serve anyone well, faculty or students.  While I’d love to see or develop such a program, I doubt we will see anyone willing to make the sort of investment required to realize such a program for many years – which is a shame, because many high-quality students that can’t move (for whatever reason; typically family-related) currently have no way to complete a Psychology Ph.D.  And that is something we should try our best to correct.