One of the biggest problems in organizational survey research, especially about sensitive topics, is that many employees do not trust that the survey responses they give will be held confidentially.

It’s led to a great number of techniques for minimizing impressions that surveys have anything at all to do with the company.  For example, if you were conducting a research project on ABC Co. about theft, it would be important not to use ABC Co. in materials.  Initial e-mail invitations should come from an independent research team with e-mail addresses unrelated to abc-company.com, the web address should not contain abc-company.com, the first page of the survey should have a color scheme unlike that of abc-company.com or any familiar company colors, a single graphic representing the company should be on the first page only to establish that this is company-sponsored (only if it really is – if the research is from a truly independent source, DON’T DO THIS), clear explanations of the anonymity/confidentiality of survey respondents should be given, and no references to the company should be made after that point, if at all possible.

Even doing this, some respondents may feel their responses are being forwarded to management.  Some researchers try to address this by asking “Do you think management will be able to link your responses directly to you?” and eliminating such people from subsequent analyses.  But thus we hit the major problem – if you thought management would be reading your responses, you’d probably say “No” to that too!

So far, we have no better solutions to this problem for survey research.  In fact, as long as there’s a person’s mind creating survey responses, this will always be a problem – at best, we can discourage such deception.  This is what leads some researchers to seek more “objective” measures, like absenteeism, units produced, weekly sales, and so on.  But there only so many things that can measured “objectively,” and many of these aren’t really that objective in the first place.

1. Attentional advice interventions direct learners to think about the application of training material to their jobs (i.e. “transfer of training”) rather than on basic memorization.
2. Meta-cognitive strategies interventions provide learners with specific strategies for internalizing information – for example, identifying where they are having trouble learning a concept and encouraging them to modify their approach to learning it.
3. Advance organizers interventions are any text or graphic organizing systems to provide students with a structure for the instruction they are about to receive.  This includes outlines and diagrams.
4. Goal orientation interventions direct learners toward increasing competence/mastery rather than toward scoring well or passing a test.
5. Preparatory information interventions give learners information about the training they are about to receive ahead of time, i.e. priming learners with a cognitive structure into which to integrate future information.

They also split their analyses by the type of outcome that was of interest:

1. Cognitive learning includes increases in knowledge of facts, procedures, and other concrete information.
2. Skill-based learning includes increases in practicable skills (the integration of knowledge into practicable behaviors).
3. Affective learning includes increases in beliefs about knowledge, e.g. the answer to the question “how much did you learn?”

This produced several interesting sets of guidelines for training and educational design.

If your goal is to increase how much students learn about facts, you should use attentional advice, meta-cognitive strategies, and goal orientation interventions.

If your goal is to increase skills, you should use goal orientation, graphic advance organizers, and attentional advice interventions.

If your goal is to make your students feel like they learned something, use goal orientation and attentional advice.

Overall then, goal orientation interventions are consistently effective.  Encouraging students to learn for the sake of learning, and not to pass tests, has a positive effect in every way examined.  Convincing students not to worry about testing is easier said than done, though.

Perhaps most importantly, none of these interventions hurt.  These lists describe the practices that we have the most evidence will give a consistent and large benefit, but all of them have seem to have either a positive or neutral effect on learning.  So if you have the time, integration of all five types of interventions will certainly be an effective approach.

1. Mesmer-Magnus, J., & Viswesvaran, C. (2010). The role of pre-training interventions in learning: A meta-analysis and integrative review Human Resource Management Review, 20 (4), 261-282 DOI: 10.1016/j.hrmr.2010.05.001

A recent study^[1] by Laurel McNall and Jeff Stanton in the Journal of Business and Psychology examines this question by presenting 208 undergraduates with one of four vignettes putting undergraduates in the job of “textbook sales representative,” which carried with it a up-to-100-mile-per-day driving requirement.  The four vignettes were the result of a 2×2 design crossing perceived purpose of monitoring (for “punishment” or to improve “customer service”) with the ability to control whether or not the device is turned on (so that it can be disabled in off-hours).  There is no statistical control in that no comparison is made to non-monitoring conditions (which is a shame!).

The results indicate that student sense of privacy invasion did indeed change according to control.  If employees can turn off their location monitors, this does seem to improve outcomes.  But somewhat surprisingly, the perceived purpose did not influence outcomes – whatever the purpose of the monitoring, students interpreted the same level of invasion of privacy from the vignettes.

There were some shortcomings to this study that make interpretation tricky.  As the authors point out, students may not have perceived either monitoring purpose as “unfair,” which would have attenuated any observed relationships.  This could have been addressed by including the aforementioned statistical control, or alternatively by measuring perceived unfairness explicitly.  The study was also conducted on undergraduates and is correlational in nature – there is no way to be sure that these findings will generalize 1) to employees in real organizations or 2) to actual behavior rather than just beliefs about privacy.

So, with that caveat, we can at least give one recommendation pretty confidently: make sure that your employees have the ability to turn off any devices that monitor them.  That does raise at least one new question, however: what should an organization do when an employee turns off their monitor at a time when it should be turned on?

1. McNall, L., & Stanton, J. (2010). Private eyes are watching you: Reactions to location sensing technologies. Journal of Business and Psychology DOI: 10.1007/s10869-010-9189-y

I arrived home from Montreal yesterday, so I thought it was time to give some closing notes on this year’s Academy of Management conference.

If you’ve been following my AOM coverage, you know that I planned to live-blog on the academic sessions I attended on Monday and Tuesday.  As it turned out, the sessions I was actively participating in, my service schedule, finicky wi-fi, and the physical distance between many sessions (0.87 miles!) made that a little tricky – I only managed to actively report on one session!  The distance was an especially troubling problem – if I needed to transfer between the Sheraton and Palais des Congres with only a 10-minute gap between sessions, I knew I would arrive 10 to 30 minutes late to that session, so I decided to just drop them instead.

The three sessions that I was actually a part of went quite well and were well-attended.  The first was a PDW (professional development workshop), where Gordon Schmidt and I explained and demonstrated the use of social media in management – as a tool to stay in contact and interact with colleagues, to reach out to students, and to interact with current and future employees.  We demonstrated about a dozen technologies over two hours, from blogs to wikis to virtual worlds.

The second was a panel discussion.  In addition to Gordon and me were several interesting contributors: (1) Gerald Kain, a professor of information systems at Boston College who studies the use of social media primarily in the healthcare industry, (2) James Lynch, a vice president of communications at American Express who manages all inter-employee communication, and (3) Julia Teahen, the president of Baker College Online, the 15,000-student virtual campus of Baker College.  The discussion brought up many interesting ideas about the use of social media from both educational and organizational perspectives, and the discussion led us right up to final buzzer.

The third was a caucus where a smaller group of us discussed the use and future of social media a little more informally.

As the web and technology czar of the OB division of AOM, I was also responsible for managing the technology components of the OB Division Awards Night and Terry Mitchell‘s acceptance talk for the OB Division’s Lifetime Achievement Award.

So how many sessions did I actually get to sit in on as an observer?  Three, one of which I blogged on.  Two of these sessions discussed the use of virtual worlds technologies, but unfortunately, they were highly theoretical and focused on overall virtual systems, which isn’t very interesting to me.  I want to know how individuals respond to these technologies.  This is sometimes called “micro-organizational” research, which is also typically the focus of industrial/organizational psychology.  There were a few promising bits and pieces in this vein, most notably the work of Tara Behrend, but overall, the Academy was a little light on social media and virtual worlds research this year.  I suppose it’s good that my own social media research is getting closer to publication!

Next year’s Academy is in San Antonio, TX.  I’m not sure who decided Texas in August was a good idea, and rumor is that the buildings will be as spread out as they were in Montreal.  But hopefully next year, I’ll at least get to a few more sessions!

The following is a live-blog/log of Twitter messages from the first day of academic material at the AOM 2010 conference.  My service commitments were pretty high this day, so I only managed to sit in on one session – it was pretty interesting though.

10:28:47 AM: Chairing measurement symposium… Interesting stuff

11:59:31 AM: In Sociomateriality and Virtual Worlds

12:00:58 PM: Missed first paper (thanks shuttle!) but looked like a basic demonstration of SL anyway

12:04:26 PM: Hmm… If your virtual self does not agree with your actual self, what happens? #secondlife #aom2010

12:10:15 PM: Do you integrate your avatar into your identity or do you keep them distinct? #secondlife

12:14:38 PM: Some people experience dual agency with their virtual selves #secondlife #aom2010

12:21:41 PM: SL user decided to virtually be involved with multiple partners to get over 1 difficult relationship. That’s not unique to virtual #aom2010

12:26:30 PM: Speaker talking about backchannel during lectures… Briefly considered waving iPad I’m on now at her #aom2010

12:30:49 PM: Still haven’t heard specific use of VWs that can’t be done with other, cheaper, easier software… Still hoping though

12:35:23 PM: Discussant bringing up tech comparison studies…

12:37:48 PM: Getting an air of dismissal from discussant… Maybe it’s just me

12:42:39 PM: Predominant reason for an org to use #secondlife is to reduce travel costs? Ugh #aom2010

It is always the case with good conferences that there are more sessions of interest than there is time to see them, however frustrating that might be.  Especially frustrating for me personally is that service commitments cut further into the time I have available!

What follows is a list of the sessions and events I’d like to attend.  I won’t make it to all of them, for a variety of reasons.  But you can at least get a sense of what I’ll be attending (and live-blogging, when possible).  Bold entries are sessions where I will be presenting and/or chairing, which makes live-blogging those events unlikely.

First, the good. There’s wireless access in each conference hotel.

Second, the bad. It’s very limited in users, at least in the Sheraton. Gordon Schmidt and I gave a professional development workshop today on social media. We were unable to log in to the network, so we enlisted the help of a Sheraton staff member who told us that the wireless connection was limited to 50 connections. 50! For a conference as popular as AOM, that’s pretty obviously too few.

We did get the situation worked out though, and Internet access was available for us during the presentation. Although that was not the end of our technical problems.

In any case, things are otherwise going quite well! I attended the HR Jr Faculty Consortium dinner afterward and was convinced there that the full-day program tomorrow will be spectacular, so I am looking forward to that.

Given the wireless situation, I am hopeful that live-blogging Monday and Tuesday will go well, but you never really know. I plan to post a tentative schedule later this weekend – but so far, it looks like it will be a little light this year. I am participating in a panel discussion and caucus on social media, as well as chairing a session on measurement, so free time for attending presentations is a smidge hard to come by.

There’s been a noticeable lack of posts in the last two weeks, and there are a few reasons for that.

First of all, it’s the summer, and that means academics (like me) are all out traveling.  I’ve been away on various business and personal trips for the last month or so, which makes it a little hard to prepare blog posts.

Second, and more importantly, the Academy of Management [AOM] 2010 conference is coming up in a week!  This is the largest and arguably most important conference to management scholars, where the latest in academic management research is presented.  This year, I am participating in three major presentations: 1) a professional development workshop teaching attendees about social media and its potential, 2) a panel discussion with several prominent management folks discussing best practices and challenges associated with deploying and handling social media in actual organizations, and 3) a caucus where Gordon Schmidt and I will be discussing social media in a much more open format with anyone who happens to stop by.

I am also the technology czar for the Organizational Behavior division of AOM, which means I have service responsibilities while I’m there too.  So if you happen to be attending any official OB division AOM events, look for the people in executive committee dress (which I believe will be blue polo shirts this year!) and say hello!  I will most likely be the man behind the camera.

As I did during SIOP 2010, I plan to live-blog my experiences and the presentations that I attend.  I’ll also be posting a tentative schedule in the next couple of days so that readers will know roughly what to expect.  The conference is in Montreal.  Because I don’t have an international phone plan, that unfortunately means that I will be tethered to wireless Internet access points.  As a result, live-blogging may not be possible – for now, we’ll just hope.

Thus an uneasy truce was struck.  Fine scientific minds would post at ScienceBlogs with the understanding that their words were not to be censored in any way, such that the integrity of the writers could be maintained.  Seed got what it wanted through advertisers, while the writers got exposure.  This is a pretty unusual arrangement for a publishing organization; imagine the New York Times telling its reporters that they could put anything into print that their hearts desired.  The reason it seems peculiar is that this more closely represents the academic model of publishing – however controversial a finding might be, it should be published if the scientific methods backing it is sound.  It was only within this model that scientists agreed to work for Seed Media Group.

Then the scandal erupted.  A corporate sponsor, PepsiCo, decided that it wanted Seed to create a ScienceBlog dedicated to food science.  It was originally to be written by an independent blogger (presumably only associated with the Pepsi name), but for some unclear reason, this did not happen.  Instead, it was set up such that Pepsi employees would write the blog – a big no-no in the academic culture I described above (the words “corporate shill” were thrown around several times).

Previous to the scandal, I had assumed ScienceBlogs operated like ReseachBlogging.org, an organization with which this blog associates.  The purpose of ResearchBlogging.org is to establish a central hub by which people interested in peer-reviewed science can connect, and it works quite well.  But ResearchBlogging.org is just that – a hub and a loose affiliation; I don’t chat with other ResearchBlogging.org folks except when I want to comment on their writings on their own blogs.  At ScienceBlogs, there’s something more closely resembling a community of coworkers.

That community in combination with academic expectations of openness and the academic tendency to analyze deeply creates what I would term a “perfect storm” – members of the community are angry and more than willing to reveal probing thoughts on the scandal, offering independent observers an uncommon glimpse deep into culture of this organization.

Here are some examples:

• Prominent ScienceBlogs member PZ Meyers gives this hot-headed assessment, with rich critical metaphor, demanding that organizational authorities comment on the problems ScienceBlogs is experiencing.  This is, in a sense, a demand for informational justice – the ScienceBlogs community has been wronged, and PZ demands that their corporate overlords provide a response.
• Blogger Coturnix gives this detailed description of the problem and his own rationale for separating from ScienceBlogs.  In an organizational sense, we can explain this as a matter of person-organization fit.  Coturnix does not feel that the values of the organization (Seed) match his own any more, and as a result, is leaving the organization.  But the amount of detail given for this decision is an incredibly interesting look into what turns turnover intentions into actual behavior.
• Former SEED employee John Pavlus minces no words, stating that “the sooner that…SEED Media Group permanently shuttered, the better off science journalism/communication would be.”  That’s a lot of hostility, but not directed at how the employee himself was treated.  Instead, his concern is for how the organization is affecting the world.  How many of you work for an organization whose actions you vehemently oppose, insofar as you would more critical of their effect on society than their effect on you?
• Gaia discusses her choice to publish unflattering information about Seed in the Guardian, describing how she an article she wrote was pulled because it criticized an advertiser.  She withheld this information until now, which makes this an interesting case study of after-the-fact whistleblowing.  Essentially, a former employee kept her mouth shut about what she perceived as biased practices, only to make a big reveal some time later.  Many companies fear exactly this.
• Emily Anthes reports a similar experience, though in a much shorter format.  Instead of censorship, this former employee was asked to write a positive piece about an advertiser.  Though we don’t have much information due to the short format, we can assume that this was again an issue of P-O fit.  This employee was asked to perform what was to her an unethical act, and as with Gaia, only decided to come forward with that information much later.

The debacle has certainly brought much about Seed into the light that I would never have learned otherwise, and almost all of it is provided by current and past employees.  What exactly did Seed do to rouse so much ire over such a long period of time, and how can others avoid these mistakes?  Certainly there are some lessons here to be learned for any organization.

Details are sparse, but this quote gives a little information:

“The camera can distinguish a person’s sex and approximate age, even if the person only walks by in front of the display, at least if he or she looks at the screen for a second,” said a spokesman for the project.

If data for different locations is analysed, companies can provide interactive advertisements “which meet the interest of people who use the station at a certain time,” the project said in a statement.

That makes it seem like the plan is to collect demographics data from the billboard, split that information by time, and then change the digital display on the billboard by time of day.  This is a clever approach to getting better outcomes from billboards, but there are some privacy concerns here.  While the company assures that only the demographic data is collected (photos are not retained), there is no way for passer-by to know this definitively.

The Daily Nation article draws parallels to Minority Report, in which advertisements were presented that customized their content and purchasing suggestions to each individual nearby, but that’s a bit far-fetched.  First, that doesn’t seem like the goal of the advertisement company here and second, that technology is currently not feasible, although it likely will be somewhere in the mysterious future.

I bring this up because I just was denied permission to conduct a video-based job analysis of an organization that I am working with.  The idea was to collect video-based critical incidents of customer service encounters and use them to develop a training program.  The videos would not be seen by administration and would not be used for administrative purposes (i.e. raises, promotions, firings, etc).  Even so, the organization felt that this was an invasion of employee privacy and would ultimately create more problems than it would solve.

Clearly, the balance between privacy and technology-enhanced data collection is becoming increasingly tricky, and there are several concerns moving forward.  What will employee/consumer reactions to this violation of privacy be?  Will there be attitude or behavioral changes?  Would real-time” advertisements be considered the same or worse than this kind of targeted advertisements?  Will there be cohort effects, i.e. will people born with such advertisements even give such advertisements a second thought?  As long as consumers get the products that will benefit them the most, will they even both to ask questions about why it’s being offered to them?

So here’s looking toward a bright and glorious future.  Just don’t eat the soylent green, however compelling the ads might be.