Happy Turking friends!
]]>I’d like to share a solution for collecting linked/dyadic data from a simple online survey. The following instruction is specific to the opensource LimeSurvey application, but could be used as a template for expansion to other survey platforms. Notably, LimeSurvey is free.
[Your name]
Now you can collect dyadic data using LimeSurvey. Good luck and please share your experiences, improvements, extensions, and/or concerns.
Image credit: https://3.bp.blogspot.com/V0cHqZl3fOA/VdUL4zegiNI/AAAAAAAABAM/oCmKZPukXsc/s640/APIM%2Bspecificis.png
]]>Do you collect occupation or job titles and recode to O*NET? If you do, you know the process is tedious and timeconsuming. Multiple raters should be used and even in the best cases, unnecessary error is added by the recoding process.
In an effort to let survey respondents decide on the closest O*NET occupation themselves, I’ve come up with the following solution. This should integrate with all of the major survey services (they just need to allow scripts to run within the survey). I use this with LimeSurvey personally. Please share any improvements!
Embed the following as script within a short answer/free text question:
<div id="realcontent"> <div id="content"> <h3 style="textalign: center;"><span style="fontsize:18px;"><span style="color:#000000;"><u>READ THESE INSTRUCTIONS CAREFULLY BEFORE ENTERING ANY TEXT!</u></span></span></h3> <p><span style="color:#000000;"><span style="fontsize:14px;">Follow the steps below to select an occupation from a standardized list that is the <em>best </em>match to your own job. You will copy a code from the search results to complete this item.</span></span></p> <p><span style="color:#000000;"><span style="fontsize:14px;">Here's how it works:</span></span></p> <ol> <li><span style="color:#000000;"><span style="fontsize:14px;">Enter your job title in the search box and <strong>click</strong> to search a list of standardized occupational titles.</span></span></li> <li><span style="color:#000000;"><span style="fontsize:14px;">Search results will open a new tab in your browser so you don't have to worry about the survey closing. <em>You might need to allow popups from this site.</em></span></span></li> <li><span style="color:#000000;"><span style="fontsize:14px;">Scan through the occupations listed from your search. Feel free to click on any of them for more information. Once you decide on the best matching occupation copy its code and return to this page. <strong>The code is listed to the left of the occupation name.</strong></span></span></li> </ol> </div> </div> <div class="searchwrapper" style="background:#FFFFFF; padding: 20px; margin: 20px; color: #2C3E50;"><span style="color:#000000;"><input class="tftext" id="tftextinput" maxlength="120" placeholder="Enter Your Job Title..." size="26" style="border:1px solid #233140;" type="text" /><button id="tfbutton" style="marginleft:8px;" type="button" value="Search O*NET Occupations">Click to search for a standardized occupational title</button> </span></div> <script type="text/javascript" charset="utf8"> $(document).ready(function() { $('#tfbutton').on('click', function(e) { e.preventDefault(); var b = $('#tftextinput').val(); window.open("http://www.onetonline.org/find/quick?s="+b) }); }); </script>
Now, make another short answer/free text question to be displayed on the same page that says something along the lines of:
“Now paste or type the occupation code in the box below. You must enter the code exactly as it appears online (i.e, include hyphens and decimal points and don’t add spaces before or after the code).”
For this answer you should validate that it follows an O*NET code format (e.g., ######.##). The specific validation text for use in LimeSurvey is: /^[15][09][09][09][09][09].[09][09]$/
Click here for a working example of the search box.
]]>I’ve also uploaded tracks to the Sound Foundry media page.
]]>For those at SIOP and interested, I’ll be presenting “A ThreeDimensional Examination of Misfit Asymmetry for Interests and Job Satisfaction.” The presentation includes an introduction to a new response surface method (“Response Surface Sampling Methodology”) that is well suited to crossvalidation and accounts for areas of misfit. Session details are below. Hope to see you there!
Day  Time  Duration  Room Name  Program ID/Format 
Thursday  12:00 PM  80 minutes  205 A  35/Symposium/Forum 

You can stream the album free on Spotify or buy the album. For the record, you can read the full album notes online at the buy link, including descriptions of the scriptural backdrop for each song. I’ve uploaded several of the tracks to the Sound Foundry media page.
]]>Sounds good, right?
Bayesian priors allow crossloadings and residual covariances of SEM’s to vary a small degree (i.e., replace exact zeros with approximate zeros from informative, smallvariance priors) and be evaluated (see Asparouhov, Muthén, & Morin, 2015; Muthén, & Asparouhov 2012). The researcher can thereby discover whether 0 crossloadings likely exist in the “true” population model, given their data, and refine their model accordingly.
I recently had the opportunity to apply this technique to a bifactor model for a new scale that had previously only been subjected to a traditional CFA. The model had strong theoretical support, but fit indices and some loadings were not supportive of the general factor as theorized. These conditions provide a perfect opportunity to use Bayesian CFA (BCFA) to refine the model for crossvalidation.
MPlus 7 was used to specify and test a model where crossloadings were assigned normally distributed priors with 0 means and variances of .01. The technique is not difficult for those familiar with CFA in MPlus. Working from a typical CFA, the researcher need only:
ANALYSIS: ESTIMATOR = BAYES; !Uses two independent MCMC chains PROCESSORS = 2; !To speed up computations if you have 2 processors FBITERATIONS=15000; !Minimum number of Markov Chain iterations
f1 BY y1y3; f2 BY y4y10; to f1 BY y1y3 y4y10; !Crossloadings f2 BY y4y10 y1y3; !Crossloadings
f1 BY y1y3 y4y10 (f1xlam1f1xlam7); !Crossloadings (with assigned labels) f2 BY y4y10 y1y3(f2xlam1f2xlam3); !Crossloadings (with assigned labels)
MODEL PRIORS: f1xlam1f2xlam3~N(0,0.01); !You can list across factors if ordered
Naturally, identification and metric setting needs to be addressed first (for example, MPlus sets the first loading of a factor to one by default, but you may want to free that and instead set the variance of each factor to 1 instead). Yet, hopefully this illustrates the ease at which crossloadings can be assigned Bayesian priors. For those interested in assigning residual covariances prior distributions the concept is very similar. A full example is given in the Asparouhov, Muthén, and Morin (2015) article.
Now for the beauty (i.e., application) of BCFA. Crossloadings were assigned a normal prior distribution with mean 0 and variance of 0.01. Were the actual loadings 0 given our data? The prior and posterior distributions for each loading can be viewed to verify this (choose “Plot” > “View Plots” in MPlus) or the confidence intervals for the crossloadings can be examined in the MPlus output. Below is an example of a crossloading for an item I analyzed:
First, examine the prior distribution (made up of 15,000 iterations):
This looks right, the mean is essentially 0 and variance (Std Dev squared) 0.01 as specified.
Now, did the distribution change given our data? Check the posterior distribution to find out (or the confidence intervals for the crossloadings in the MPlus output):
It appears the distribution did change. Given our data, the posterior distribution does not include 0 (for a 95% confidence interval). This suggests that the specific crossloading should be freed.
This simple illustration is just one way BCFA (and BSEM) adds value for scale and model development. I refer you to the linked articles for more. Let me know if you’ve had a chance to apply the technique and what you learned.
References
Also enjoyed our Academy of Management paper session–The Role of Dark Side Personality in Work and Career Contexts: Exploring New Avenues. Fantastic presentations from Jasmine Vergauwe, Bart Wille, and Jeff Foster… and many thanks to Benjamin Galvin for his facilitation and insight as discussant and my coauthor, Emily Grijalva, for her input and direction throughout.
As promised: here are my slides from the session
And my posts on using MTURK with a time lag start here.
]]>
A prominent research interest of mine is assessing personvocation fit and its relationship to work outcomes, such as job performance. Polynomial regression (PR) and response surface methodology (RSM) are ideal methods for measuring personvocation. Reasons for the superiority of PRRSM are numerous, but not the point of this post. The interested reader should see Jeff Edward’s writings here and here for rationale. The point of this post is to offer highlevel, practical guidance on how to apply PRRSM to fit questions. So, without further ado:
PRRSM Guide for Evaluating PE Fit
I hope this helps motivate those with interesting fit questions to think about using PR and RSM. This has largely been aimed at examining hypothesized relationships, but you are free to do exploratory analysis as well. There are challenges to accomplishing the necessary analysis and interpreting 3D plots, but the method is well worth the effort!