Running studies on unforgettable.me

The first step to becoming a researcher on the unforgettable.me platform is to sign up for an account as a research participant. Once that is done, login to your account and navigate to the Marketplace. On the right hand side of the screen you will see a button entitled “Request researcher account”.

Click on this and the following dialog message should appear:

Once you have been approved, a new section titled “Researcher” will appear in the left hand side panel. This section contains four options – Account, Load Credit, Projects and + Project.

Click on Account and you will see your current researcher balance – the amount of money that you have loaded onto the system but have yet to spend. As you create projects you will indicate how many participants you want and how much they will be paid. The liability you have incurred from these projects (both to pay participants and unforgettable.me fees) will appear in the Funds held for liability field. The Available funds are those that you still have available to devote to new projects.

To add new funds to your account, click on Load credit. Right now this tab redirects you to the contact page so you can send a message to us to arrange payment. In the future, you will be able to add funds directly here.

Creating a Research Project

We will create a project that requires Twitter posts from participants. To demonstrate, we will connect a Twitter feed to this account using IFTTT and then add a project that queries for this data. We have another tutorial on how to connect IFTTT applets. Once this is done, we post some tweets;

which will then appear in the unforgettable system:

Now hit the + Project tab and fill in the form as follows:

The title will be the label used to identify the project.

The researcher list allows you to add the email addresses of the people who will need access to the data. These must be registered users of the unforgettable.me system. If they are not the person icon will appear red, otherwise it will be green. For the purposes of the demonstration, we will just add the current account as a researcher.

In the next section, you can add a series of queries that define the participant data that you wish to access. The first field is the string to search for in the participants data. We could have typed “Twitter” and this would have isolated the events of interest in our demonstration. However, if the participant had other events such as emails that also included the term Twitter these would also be included. For this reason, we use the tag __IFTTTtwitter__ which is included in all events that come from the IFTTT twitter applet.

The date range over which we would like the query to extend must also be specified. The range can be entered between two absolute dates. However, it is often more convenient to specify a relative date range. In this case, the time period for the data collection will count backwards from the time at which the participant joins the project. In this case, we are looking for Twitter events that occurred in the four weeks prior to them joining.

In rare cases, it is also helpful to be able to specify a buffer between the time when they join and the end of the data collection period. For instance, in memory experiments one might want to set a retention interval, so that there is a gap between the test and when the events occurred. The relative date buffer allows one to do this, although most of the time it is left blank.

You can also specify how many events of this kind the participant must have in order to join the study. You need to estimate the rate at which events will be collected and set this to a reasonable number given the time period that will elapse. For instance, if you were collecting __App__ events using the unforgettable.me smartphone app for a month, you might estimate that there would be 24*30 = 720 events. However, the participant might have some circumstances in which they turn off the app for privacy reasons. So you might say they need to have at least 600 events. In our case, we have only 2 events in our data store, so we indicated that the participant must have more than 1 event.

Note you can set up multiple queries to allow you to specify more complicated data sets.

The next field is the description of the project. When participants are browsing studies to decide whether to join they will see this text, so it is important that it explains the point of the research and outline what will be required of them. Here is a sample from an event segmentation project that we have run:

We would like to invite you to participate in a project on people’s ability to decide when one event finishes and the next begins in daily events. This research aims to determine how people decide what constitutes an event and which kinds of input people are using to determine when event boundaries occur. We will collect data as people go about their lives using the sensors available on their smartphones, through the use of two smartphone apps – the SEMA3 app and the Unforgettable Research Services app. Also, we will access data from the electronic calendars through IFTTT.

Maximum pay is capped at $91.50. Pay advertised above of $18 is for the Unforgettable App and Calendar data. Payment for SEMA events depends on the number of surveys completed ($36.75 – $73.50), the payment for these SEMA surveys will be calculated by the researcher after the study and paid as a bonus using the Unforgettable system.

The next two fields specify how many participants you would like and what you intend to pay them for the use of their data. We provide guidelines for appropriate payments, but you are free to decide what you will offer. For the purposes of demonstration, we will set the payment to $0.

It is often the case that there will be additional tasks that you wish the participants to complete after they have joined. For instance, in memory experiments participants will often be given memory cues and asked to recall events from their lives. Alternatively, you may require them to complete personality or time use surveys. The Allocated time indicates how long they have from when they join to complete the task. Note it is specified in minutes so if you intend to give them several days this number will be large.

The project expires time is how long from the time the study is constructed to when it expires and no more participants will be collected. This number can be quite large. Suppose you wanted to run the study for three months, then you would set this value to 3*30*24*60 = 129600 minutes.

Sometimes you would like to ensure that two projects you run do not have overlapping participants. Exclude Project IDs allows you to indicate to the system that subjects who participated in these projects should not be sought for this project.

The Requires consent checkbox indicates if the participants should tick a box to indicate that they consent to participate in the study. When participants click consent a “Consent Event” is added to their dataset. The Associated files dialog allows you to upload pdf files that the participant will see prior to providing consent – such as briefing sheets and consent forms.

Once you have finished completing the form click ‘Submit project request’:

Once your project has been approved, it will appear in the All projects tab:

You can then click to provide informed consent:

And then click join project to become a participant:

Because there are no additional tasks to complete, you will see the project complete label straight away. If there were additional tasks, then you would be directed to those at this point.

Because the project required only one participant, and that participant (you!) has now completed the study, when we navigate to the Research Projects tab we see two buttons. You can download the data for entry into other analysis packages by clicking on the Export button.

If the data set is large, it can take some time for it to be packaged ready for export. When it is done, an email will be sent to you with a link:

Clicking on this link will download a file called export.csv to your computer. Below, this file has been loaded into Google Sheets for further analysis.

And that is it! You have now successful executed your first project. If you have any questions you can contact us at www.unforgettable.me/contact.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s