I'm creating a survey for my master's thesis, part of which randomizes participants seeing one of two sets of images. Surveys, Online It will ensure that respondents are randomly assigned to your three conditions evenly instead of just randomly. The survey flow would look like this: Within the ConditionA/ConditionB blocks pipe, "This is Condition ${e://Field/condNum}". I would guess that you practiced your survey a bunch before launching, using the survey preview option. Is using this feature still appropriate for randomizing participants in a controlled experiment though? I set the number of elements presented to a participant to 1. New Hire I noticed that the blocks it do not seem to randomize. Seems to work as well! It works by keeping count of how many times it picks each item in the randomization. New Hire I don't quite understand it, and I'm unsure whether I should use it or not. You need to have four separate embedded data blocks under the randomizer that each assign a different value to CONDITION. Software, Website & App Community. Engagement Software, Website & App Software, 360 I am having trouble understanding the "evenly present elements" option. Surveys, Onboarding & New Hire Pulse Customer However, the estimated response time and the progress bar do not take that into account. Software, Employee Community. Software, 360 New Hire Software, 360 Is there anything you did in the survey builder. Customer Experience Management (CXM), Employee But it might come up heads 98 times. You have two problems in your image. New Hire However, after they have finished one of these randomised conditions, I actually want the participants to be shown the instruction block of the condition that they haven't done yet, after which they will be branched to that condition's tasks as well. Assuming 15 participants this is how I want them to be distributed: It should work in the way I set the survey flow, however it seems that the block 5a,5b,5c do not get displayed. So my script was targeting the previous question always (contains('td', 'Unable to rate') had a length of 2 instead of 1), Customer Experience Management Feedback, Customer Survey Surveys, Online This is helpful. After ever one of those 22 blocks you update the counter by one and check it's value. In each group, the first block is the sentence you want to display and the second block is the actual condition. Employee Use a combination of loop and merge and display logic. Software, Website & App Pulse New Hire Pulse Feedback, Voice of Feedback, Voice of Software, Employee Employee Customer E.g. Thanks again! Having a simple window.alert("hello world"); in onLoad does show, but not my code: I thought it has something to do with the elements not having been loaded yet, so not being available. If I select "exclude from progress bar calculation" the block that is actually the one presented will be excluded, but I would like to exclude the other 2 blocks which are not presented to the participant. Or do I have to use JavaScript (or something else piped fields, etc.) Management, Qualtrics Surveys, Onboarding & (CXM), Employee Software, Website & App Feedback, Customer Survey Employee Community. Engagement I am designing a survey with 3 conditions and I'd like to randomize the participants into those conditions. Software, Website & App Reputation Customer But that doesn't explain why it does work for the first block. (CXM), Employee Feedback, Customer Survey See image below. with addition to what akshay has said, Yes, you should use it. But the survey automatically skips all questions, https://community.qualtrics.com/XMcommunity/discussion/comment/5620#Comment_5620, Customer Experience Management Software, Website & App Surveys, Online

Customer Experience Management Feedback, Customer Survey I want to nest two randomizer but I do not know how it works; look at the image: I want that each participant will be randomized to one of the 5 conditions - However I want that participants that goes into condition 5: book chapter would be randomized to one of the three conditions: condition 5a, condition 5b, conditon 5c. (CXM), Employee New Hire Employee Surveys, Onboarding & Qualtrics checks the number of times each element has been presented, shows the one which has been presented least, If more than one element has been presented the same number of times (lowest), it does a random draw to present one, It does this till all elements have been presented an equal number of times. (CXM), Employee New Hire Software, 360 I have been looking at Branch logic and so far I'm stymied. I need to use custom code; however, I don't know anything about coding. Surveys, Onboarding & I have 3 conditions that will be presented in different random orders. Management, Qualtrics Management, Qualtrics So far so good. Customer in the embedded data updates after each block? Surveys, Online I'm making a survey for my dissertation. Anyone any idea what can be happening here? Community. What assumptions do I have to make to argue this is not causing any biases? Reputation New Hire Customer Experience Management Customer I previewed the survey a few time and to see it randomizes. Management, Qualtrics Short answer: Use a embedded data field with as a counter. On average, yes. Surveys, Onboarding & and check below attached screenshot.. I want to randomly present 5 of these blocks, then ask if a respondent would like to do more, and then present 3 more random blocks if the answer is yes (without repeating any of the blocks they have already seen). Surveys, Online Surveys, Onboarding & Management, Qualtrics Software, Employee Software, 360 Feedback, Voice of What is the best way to do this? I can't speak specifically to the Qualtrics algorithm, but typically it is done by first shuffling the set of items, then sorting them by count. Surveys, Onboarding & Surveys, Online

If, for some reason, you tested one condition less/more than the others, the randomizer will work to balance them out again and therefore send all participants to the ones that have the least. If it equals 5 you ask the See More? (CXM), Employee Since it's randomised, it's clear that it doesn't have anything to do with the block: every time I run it, it (obviously) presents the blocks in random order, and whatever block is presented first, the onLoad JS works, but for the subsequent blocks it doesn't. Feedback, Voice of You will repeat all blocks in both branches but users will only fit into one condition so they will only see each block once. Engagement Software, Website & App This feature helps guard against the unlikely, which becomes less important as the number of responses goes up. Regarding discrepancy, did you reset the count before launching the survey? It works beautifully. It does not guarantee that you will have equal numbers of completes because people could abandon the survey after being assigned but before completing, but it improves the odds. Community.

Engagement Surveys, Online Employee How can I exclude those and only take the presented block into account for the progress bar? Surveys, Online However, when i sent out this survey, all participants were given the first condition. Perhaps a better way to generate test data and check the data file for block order. Engagement Thank you very much for your help! hi, I followed your instruction. However, I noticed that as soon as I have a Randomizer in my Survey Flow, with say 9 blocks, the onLoad JS only works for the first block. Software, 360 I wrote a bit of JS and put it in the addOnLoad part, for multiple blocks. I have 22 blocks in my survey. Employee Feedback, Customer Survey Instead of nested randomization. We are evenly showing either block 1 or block 2 using the randomizer. I would also use a branching logic of if the question that is found in condition 3 is displayed, the participant THEN is shown 3a, 3b, 3c. Feedback, Customer Survey (CXM), Employee Reputation Engagement I came across a scenario where I am having 26 completes for block 1 and 14 completes for block 2 and around 14 respondents are showing in progress while randomizer is showing 51 counts for both block 1 and block 2. You also should make sure to select to reset the count of these randomizers before launching your survey (see more about this here in a previous comment of mine: https://community.qualtrics.com/XMcommunity/discussion/comment/44145#Comment_44145). Reputation Surveys, Onboarding & Pulse The JS is identical across blocks. I am setting up an experiment with 2 conditions that each participant needs to go though. Feedback, Voice of Currently I have set up a randomizer with 3 embedded data elements which assigns one of three values (1,2 or 3) to the variable "CONDITION". Customer block and if they say No you exit/continue with the rest of the survey. I've attached a screenshot of the current part of my flow. But as for the main question, does it have an impact for a large enough sample, I don't think it should make a difference. Community. Software, 360 Feedback, Voice of Software, 360 Okay so it turns out that onLoad is working normally, but that Qualtrics -- when in a Randomizer block -- loads 2 questions: the previous one, in case people want to go back (I assume), and then the current one. Reputation Reputation The estimated time response shows the time needed to answer all questions (out of all three, not just one block) and the progress bar skips immediately to 66.66% after the introduction (because it takes the two other blocks into account). Because CONDITION is always 4 since it is the last one assigned. Reputation Therefore I used the randomizer, stating to present only one out of three. New Hire Could you Group your blocks as below? Then randomly present 1 of the 4. Pulse That means, each participant is seeing EVERY block. @MSobotta - Thank you! How do I go about resolving this? Software, 360 New Hire Customer Experience Management Does anyone deal with the same problem? Management, Qualtrics Customer Feedback, Customer Survey The randomizer logs each of these as a case, and therefore keeps track of which condition your test was sent through. Feedback, Voice of I will use it then. Customer Experience Management Employee Software, Employee In other words, I want to fix blocks containing these sentences under the Randomizer. Qualtrics isn't aimed at scientific studies, its more for business purposes. Customer

I would not use js it's just not necessary. create three flags and set it to 0 Engagement (CXM), Employee New Hire Software, Employee Feedback, Customer Survey What do i need to change in my survey so that it'll be randomized. I have added randomizer with a few blocks on the survey. How would I alter my flow so that it first shows one instruction block randomly, followed by the appropriate tasks, and then the other instruciton block, which is then also followed by the right task's block? I figured it out - and the answer is in this thread, for anyone who would like to know later: https://community.qualtrics.com/XMcommunity/discussion/7088/hi-i-want-to-give-a-number-for-each-question-when-randomizing-block. For both the scenarios, the answer is yes, as soon as the respondent reaches the randomizer, the count gets incremented. Customer Management, Qualtrics Community. Surveys, Onboarding & Customer You should click 'edit count' on the randomizer and make sure all conditions have a count of 0 before launching your study. For the counter, can I simply start by defining embedded data as "counter=0" and then in future embedded data flow updates say "counter=counter+1" and have that work? Customer Experience Management Reputation Reputation I finally had to use Javascript to increase variable count :D My problem was solved. The order of these conditions needs to be randomised for each participant. Thanks, @phbern, @TomG, @ahmedA for answering. You need to change this to be 1 in each randomizer. Software, Website & App Within the randomizer you have 22 groups that contain your block, a counter, your "like more?" I ended up using groups and randomising those. Feedback, Voice of Pulse I'm still not fully convinced about what it means for the scientific rigor of an experiment. Pulse Software, Employee Software, 360 Pulse thanks for the response. It will reduce the variability in how many respondents are assigned to each condition. Engagement @juwort Put an embedded data field in your Randomizer and then change your branches so that they are triggered on the value of your embedded data field. Software, 360 block, and end/rest of survey. Without evenly present: Qualtrics does a random draw and presents one element to the respondent. You can uncheck evenly present elements and then try out the randomizer. Pulse Surveys, Online It reduces the need to over sample and makes statistical analysis better since you are less likely to run into issues of small sample sizes if you cut the data by other criteria. I tried to publish the survey and look at it then, but same problem. Feedback, Voice of Management, Qualtrics Is there any reason you can suggest behind such discrepancy. Community. Customer Feedback, Customer Survey Community. Community. There should be two branches that contain the order of the blocks for the condition chosen by the randomizer. Employee Great, thank you for your insights. (CXM), Employee I set a variable count is 0, but it doesn't work when running the survey. Pulse Use a combination of groups, arithmetic operations, and piping. (CXM), Employee

Engagement Feedback, Voice of Before each condition, I want to show a sentence like "This is Condition 1"; "This is Condition 2" according to the order the condition appears. Customer Experience Management Customer Feedback, Voice of Software, Employee Management, Qualtrics I actually tried group, embedded data, and display logic. I can't just have the randomiser present both the instruction blocks, as these will then both be shown before participants are branched to their actual tasks. Software, Website & App Is there something I'm just not seeing? Customer Experience Management Pulse My embedded data is shown below. Once that's done, as many items as are needed are picked from the top of the list. For now I put both the instruction blocks in a randomiser and put the branches that will then guide the participants to the appropriate tasks outside of the randomiser (to avoid randomisation of my branches as well). Software, Employee Surveys, Onboarding & Setup in the survey flow looks fine.

I have a problem that cannot be solved by existing Qualtrics functions. So after a recipient sees a block the counter goes up one. If you don't mind putting all the conditions in one block. Is there a way to change this? Surveys, Online Employee (CXM), Employee Management, Qualtrics If the respondent terminates before the randomizer section, does still counts counts will increase in randomizer for such ids. What have I done wrong? Employee Software, Employee Software, Website & App Reputation How many times have you tested? Community.

Software, 360 From my small testing of the randomizer (3-4 experiments, 10-12 draws, 2-4 elements in the randomizer), here's the difference that I noticed. Community. there is a 33.3% chance in my case, then in the end there will be roughly the same amount of participants in each condition, without having to execute any feature that "evenly presents" the options, right? (CXM), Employee how does the evening out work in qualtrics? I doubt you'll find a more detailed explanation of these anywhere. If any respondent has passed randomizer section but does not completed the survey, in that case will counts increases in randomizer or it will only increases once survey will be completed. Software, Employee

Surveys, Onboarding & Management, Qualtrics Community. Software, Website & App But should I check the "evenly present elements" option? Reputation Surveys, Online

Feedback, Customer Survey Surveys, Online I have separate instruction blocks for each condition and then separate blocks for the actual tasks (these blocks use a loop and I don't want the instruction to be repeated for each loop, hence the separate blocks). https://www.qualtrics.com/community/discussion/comment/31170#Comment_31170, https://www.qualtrics.com/community/discussion/comment/31171#Comment_31171, https://www.qualtrics.com/community/discussion/comment/34518#Comment_34518, Customer Experience Management Feedback, Voice of Feedback, Customer Survey Software, 360 Engagement Software, Employee Can you please answer my below queries regarding randomizer. Feedback, Customer Survey Engagement if you can also do following using randomizer. Surveys, Onboarding & In both randomizers you have said you want the randomizer to randomly display "6" and "3" (respectively) to each participant. Please support me on this request. Software, Website & App Reputation The survey will entail a participant being allocated to one of four vignettes randomly, and then filling out a set of questions. If what condition one is assigned to is randomized, i.e. For my dissertation, i need to randomize participants to one of three conditions. You might need several tries to see the order flip. Software, Employee Engagement I'm still figuring this stuff out.). This format works perfectly fine for the other randomization element I have earlier in the study. Management, Qualtrics Management, Qualtrics Software, Website & App Reputation Engagement Is there also a more profound discussion of this somewhere? Pulse Customer Surveys, Onboarding & New Hire Employee (CXM), Employee I've tried setting up the randomizer by using embedded data as shown below: But for some reason on my test responses it solely allocates participants to Condition 4. If you are wanting it to choose a block to show then you need to change the number from 2 to 1 in your randomizer and check Evenly Present Elements.

Surveys, Online Long answer: You create a randomizer in the survey flow and present 8 items making sure not to repeat. If I throw a coin 100 times, it's gonna be about 50 times heads and 50 times tails. I have three blocks in my survey but only one will be presented to each participant. https://community.qualtrics.com/XMcommunity/discussion/comment/44145#Comment_44145, Customer Experience Management Software, Employee Feedback, Voice of Customer Experience Management Feedback, Customer Survey Pulse Employee Employee