TL;DR: Do WaitPages ignore before_next_page? If so, how can I assign values to a participant variable?

I have a multi-app project. In App 1, participants cast a vote. In App 2, I want to group them by arrival time, compute the group-level vote share, and assign the vote share to a participant variable for use across future rounds. I also want to assign a random treatment variable to each group and set it as a participant variable. The core of the second app looks like this (full MWE below/attached):

class Subsession(BaseSubsession):
    pass
def creating_session(subsession: Subsession):    
    treatments = itertools.cycle(['X', 'Y'])
    for group in subsession.get_groups():
        group.treatment = next(treatments)
class Group(BaseGroup):
    treatment = models.StringField()
    voteshare = models.IntegerField()
class Player(BasePlayer):
    pass
def get_votes(group: Group):
    # Compute outcome of vote
    players = group.get_players()
    votes = [p.participant.vote for p in players]
    group.voteshare = sum(votes)
class WaitVote(WaitPage):
#    group_by_arrival_time = True
    after_all_players_arrive = get_votes
    @staticmethod
    def before_next_page(player: Player, timeout_happened):
        group = player.group
        participant = player.participant
        participant.treatment = group.treatment
        participant.voteshare = group.voteshare
        
With this setup, group.treatment and group.voteshare are recorded in the data, but participant.treatment and participant.voteshare are not. If I move the code in before_next_page to the next, regular page, both participant variables are recorded correctly. Is there a way to do this directly on the WaitPage?

Second, I would also like to use group_by_arrival_time (here commented out). However, when I do this, neither of the two group variables are set. From other threads, I thought it was possible to use after_all_players_arrive below group_by_arrival time. This has left me puzzled about the correct order of creating_session, group_by_arrival_time, after_all_players_arrive, and before_next page. 

# settings.py

from os import environ
SESSION_CONFIG_DEFAULTS = dict(real_world_currency_per_point=1, 
                               participation_fee=0)
SESSION_CONFIGS = [dict(name='MWE', 
                        num_demo_participants=None, 
                        app_sequence=['vote_stage', 'main_stage'])] 
LANGUAGE_CODE = 'en'
REAL_WORLD_CURRENCY_CODE = 'GBP'
USE_POINTS = True
DEMO_PAGE_INTRO_HTML = ''
PARTICIPANT_FIELDS = ['treatment','vote','voteshare']
SESSION_FIELDS = []
ROOMS = []

ADMIN_USERNAME = 'admin'
# for security, best to set admin password in an environment variable
ADMIN_PASSWORD = environ.get('OTREE_ADMIN_PASSWORD')

SECRET_KEY = 'blahblah'

# if an app is included in SESSION_CONFIGS, you don't need to list it here
INSTALLED_APPS = ['otree']

# vote_stage/_init_.py

from otree.api import *
import random
c = cu

# Define constants
class C(BaseConstants):
    NAME_IN_URL = 'vote_stage'
    PLAYERS_PER_GROUP = None
    NUM_ROUNDS = 1
    
# Define parameters
class Subsession(BaseSubsession):
    pass
class Group(BaseGroup):
    pass
class Player(BasePlayer):
    vote = models.BooleanField(choices=[[True, 'A'], [False, 'B']], label='What do you choose?')
 
# Pages
class Vote(Page):
    form_model = 'player'
    form_fields = ['vote']    
    
    @staticmethod
    def before_next_page(player: Player, timeout_happened):
        participant = player.participant
        participant.vote = player.vote
    
# Page sequence
page_sequence = [Vote]

main_stage/_init_.py

from otree.api import *
import random
import itertools
c = cu

class C(BaseConstants):
    NAME_IN_URL = 'main_stage'
    PLAYERS_PER_GROUP = 4
    NUM_ROUNDS = 1
class Subsession(BaseSubsession):
    pass
def creating_session(subsession: Subsession):    
    treatments = itertools.cycle(['X', 'Y'])
    for group in subsession.get_groups():
        group.treatment = next(treatments)
class Group(BaseGroup):
    treatment = models.StringField()
    voteshare = models.IntegerField()
class Player(BasePlayer):
    pass
def get_votes(group: Group):
    # Compute outcome of vote
    players = group.get_players()
    votes = [p.participant.vote for p in players]
    group.voteshare = sum(votes)
class WaitVote(WaitPage):
#    group_by_arrival_time = True
    after_all_players_arrive = get_votes
    @staticmethod
    def before_next_page(player: Player, timeout_happened):
        group = player.group
        participant = player.participant
        participant.treatment = group.treatment
        participant.voteshare = group.voteshare
class VoteResult(Page):
    form_model = 'player'
    
page_sequence = [WaitVote, VoteResult]

Do it in after_all_players_arrive

Thanks, Chris. I have now (mostly) solved this by moving everything to after_all_players_arrive (see below, for future reference). 

However, I don't think it is possible to assign balanced treatments this way if also using group_by_arrival_time, is it? The itertools approach (https://otree.readthedocs.io/en/latest/treatments.html#balanced-treatment-groups) does not (seem to) work.

---

# main_stage/__init__.py

from otree.api import *
import random
import itertools
c = cu

class C(BaseConstants):
    NAME_IN_URL = 'main_stage'
    PLAYERS_PER_GROUP = 4
    NUM_ROUNDS = 1
class Subsession(BaseSubsession):
    pass
class Group(BaseGroup):
    treatment = models.StringField()
    voteshare = models.IntegerField()
class Player(BasePlayer):
    pass
class WaitVote(WaitPage):
    group_by_arrival_time = True
    @staticmethod
    def after_all_players_arrive(group: Group):
        # Set group treatment
        group.treatment = random.choice(['X', 'Y'])
        # Compute outcome of vote
        players = group.get_players()
        votes = [p.participant.vote for p in players]
        group.voteshare = sum(votes)
        # Set participant variables
        for p in players:
            participant = p.participant
            participant.treatment = group.treatment
            participant.voteshare = group.voteshare
class VoteResult(Page):
    form_model = 'player'
    
page_sequence = [WaitVote, VoteResult]

> However, I don't think it is possible to assign balanced treatments this way if also using group_by_arrival_time, is it? The itertools approach (https://otree.readthedocs.io/en/latest/treatments.html#balanced-treatment-groups) does not (seem to) work.

I've now used this workaround, just in case somebody might look for this:

class C(BaseConstants):
    TREATMENTS = ('X', 'Y')

class Subsession(BaseSubsession):
    counter = models.IntegerField(initial=0)
    
class WaitVote(WaitPage):
    group_by_arrival_time = True
    def after_all_players_arrive(group: Group):
        group.treatment = C.POLICIES[group.subsession.counter % 2]
        group.subsession.counter += 1

Hi Simon, 
thanks for the question and for the workaround you posted - it's been really helpful. 
could you quickly clarify what is C.POLICIES in the code for your WaitVote page? 
thanks!
ZW

Hi ZW,

Looks like I mixed something up in my code there. C.POLICIES should be called C.TREATMENTS. It refers to the BaseConstant define above. 

C.TREATMENTS is a vector of possible treatments. My approach indexes this vector and assigns treatment X if the group.session.counter is even and treatment Y if it is odd. If you have more than two treatments, you'd need a corresponding way to index the vector of treatments.

--Simon

Hi Simon,
Thanks for the lightning quick reply. 
I had figured out as much - but for some reason this particular workaround (with my 3 treatments and changing the mod function to mod3) does not seem to work for me. On the local demo, regardless of who I pass through first, they are all still lumped in one group with no treatment info stored in the group.treatment field.
In which class is your field for the group-level treatment information? Mine is still in the group class, but I am not sure that's the right place for it... 

And correct me if I am wrong, but the after_all_players_arrive is a session level method? So if I have a session with 12 players, would I neeed all 12 players to have arrived from app 1 in order for this to run? Or can I still form groups of 4 with the first four that arrive? 

Thanks for all your help!
ZW

And here's a snippet of my init.py file: 

class C(BaseConstants):
    NAME_IN_URL = 'slider_feedback'
    PLAYERS_PER_GROUP = 4    
    NUM_ROUNDS = 1          
    treatments = ['No', 'Positive', 'Negative']

class Subsession(BaseSubsession):
    num_groups_created = models.IntegerField(initial=0)

class Group(BaseGroup):
    treatment = models.StringField()
    [...]
    
class Player(BasePlayer):
    #Task-related fields
    [...]

#PAGES

class Count_Start(Page):
    group_by_arrival_time = True 

    @staticmethod
    def after_all_players_arrive(group: Group):
        subsession = group.subsession
        index = subsession.num_groups_created % len(C.treatments)
        group.treatment = C.treatments[index]
        subsession.num_groups_created += 1

Hi ZW,

> In which class is your field for the group-level treatment information? Mine is still in the group class, but I am not sure that's the right place for it... 

Further down in my WaitPage, I set the treatment as a participant variable. This allows the treatment to be retained across rounds. There's two steps to this. First, add 'treatment' as a participant field in settings.py. Second, set participant.treatment to group.treatment:

players = group.get_players()
for p in players:
    participant = p.participant
    participant.treatment = group.treatment

> And correct me if I am wrong, but the after_all_players_arrive is a session level method? So if I have a session with 12 players, would I neeed all 12 players to have arrived from app 1 in order for this to run? Or can I still form groups of 4 with the first four that arrive? 

With after_all_players_arrive, that would be the case. Note that in my workaround, I use group_by_arrival_time. This will form groups of size PLAYERS_PER_GROUP (i.e., 4 in your case) once a sufficient number of players have arrived at the WaitPage.

--Simon

I'm not sure but I think Count_Start should be a WaitPage so that after_all_players_arrive gets executed.

Thanks @Simon - that has been really helpful. 
And @Daniel, indeed it only works if the page is a WaitPage! (part of why it wouldn't work!) 
Thank you all for contributing - this community is brilliant!