alex_zag_al comments on Priors as Mathematical Objects

[alex_zag_al]

Here’s some Python code to calculate a prior distribution from a rule for assigning probability to the next observation.

A “rule” is represented as a function that takes as a first argument the next observation (like “R”) and as a second argument all previous observations (a string like “RRWR”). I included some example rules at the end.

EDIT: oh man, what happened to my line spacing? my indents? jeez.

EDIT2: here’s a dropbox link: https://​​www.dropbox.com/​​s/​​16n01acrauf8h7g/​​prior_producer.py

from functools import reduce

def prod(sequence):
    '''Product equivalent of python's "sum"'''
    return reduce(lambda a, b: a*b, sequence)

def sequence_prob(rule, sequence):
    '''Probability of a sequence like "RRWR" using the given rule for
    computing the probability of the next observation.

    To put it another way: computes the joint probability mass function.'''
    return prod([rule(sequence[i], sequence[:i]) \
                 for i in range(len(sequence))])

def number2sequence(number, length):
    '''Convert a number like 5 into a sequence like WWRWR.

    The sequence corresponds to the binary digit representation of the 
    number: 5 --> 00101 --> WWRWR

    This is convenient for listing all sequences of a given length.'''
    binary_representation = bin(number)[2:]
    seq_end = binary_representation.replace('1', 'R').replace('0', 'W')

    if len(seq_end) > length:
        raise ValueError('no sequence of length {} with number {}'\
                          .format(length, number))

    # Now add W's to the beginning to make it the right length - 
    # like adding 0's to the beginning of a binary number
    return ''.join('W' for i in range(length - len(seq_end))) + seq_end

def prior(rule, n):
    '''Generate a joint probability distribution from the given rule over
    all sequences of length n. Doesn't feed the rule any background
    knowledge, so it's a prior distribution.'''
    sequences = [number2sequence(i, n) for i in range(2**n)]
    return [(seq, sequence_prob(rule, seq)) for seq in sequences]

And here’s some examples of functions that can be used as the “rule” arguments.

def laplaces_rule(next, past):
    R = past.count('R')
    W = past.count('W')
    if R + W != len(past):
        raise ValueError('knowledge is not just of red and white balls')
    red_prob = (R + 1)/(R + W + 2)
    if next == 'R':
        return red_prob
    elif next == 'W':
        return 1 - red_prob
    else:
        raise ValueError('can only predict whether next will be red or white')


def antilaplaces_rule(next, past):
    return 1 - laplaces_rule(next, past)