Random functions

Hail has several functions that generate random values when invoked. The values are seeded when the function is called, so calling a random Hail function and then using it several times in the same expression will yield the same result each time.

Evaluating the same expression will yield the same value every time, but multiple calls of the same function will have different results. For example, let x be a random number generated with the function rand_unif():

>>> x = hl.rand_unif(0, 1)

The value of x will not change, although other calls to rand_unif() will generate different values:

>>> hl.eval(x)  
0.5562065047992025

>>> hl.eval(x)  
0.5562065047992025

>>> hl.eval(hl.rand_unif(0, 1))  
0.4678132874101748

>>> hl.eval(hl.rand_unif(0, 1))  
0.9097632224065403

>>> hl.eval(hl.array([x, x, x]))  
[0.5562065047992025, 0.5562065047992025, 0.5562065047992025]

If the three values in the last expression should be distinct, three separate calls to rand_unif() should be made:

>>> a = hl.rand_unif(0, 1)
>>> b = hl.rand_unif(0, 1)
>>> c = hl.rand_unif(0, 1)
>>> hl.eval(hl.array([a, b, c]))  
[0.8846327207915881, 0.14415148553468504, 0.8202677741734825]

Within the rows of a Table, the same expression will yield a consistent value within each row, but different (random) values across rows:

>>> table = hl.utils.range_table(5, 1)
>>> table = table.annotate(x1=x, x2=x, rand=hl.rand_unif(0, 1))
>>> table.show()  
+-------+-------------+-------------+-------------+
|   idx |          x1 |          x2 |        rand |
+-------+-------------+-------------+-------------+
| int32 |     float64 |     float64 |     float64 |
+-------+-------------+-------------+-------------+
|     0 | 8.50369e-01 | 8.50369e-01 | 9.64129e-02 |
|     1 | 5.15437e-01 | 5.15437e-01 | 8.60843e-02 |
|     2 | 5.42493e-01 | 5.42493e-01 | 1.69816e-01 |
|     3 | 5.51289e-01 | 5.51289e-01 | 6.48706e-01 |
|     4 | 6.40977e-01 | 6.40977e-01 | 8.22508e-01 |
+-------+-------------+-------------+-------------+

The same is true of the rows, columns, and entries of a MatrixTable.

Setting a seed

All random functions can take a specified seed as an argument. This guarantees that multiple invocations of the same function within the same context will return the same result, e.g.

>>> hl.eval(hl.rand_unif(0, 1, seed=0))  
0.5488135008937808

>>> hl.eval(hl.rand_unif(0, 1, seed=0))  
0.5488135008937808

This does not guarantee the same behavior across different contexts; e.g., the rows may have different values if the expression is applied to different tables:

>>> table = hl.utils.range_table(5, 1).annotate(x=hl.rand_bool(0.5, seed=0))
>>> table.x.collect()  
[0.5488135008937808,
 0.7151893652121089,
 0.6027633824638369,
 0.5448831893094143,
 0.42365480398481625]

>>> table = hl.utils.range_table(5, 1).annotate(x=hl.rand_bool(0.5, seed=0))
>>> table.x.collect()  
[0.5488135008937808,
 0.7151893652121089,
 0.6027633824638369,
 0.5448831893094143,
 0.42365480398481625]

>>> table = hl.utils.range_table(5, 5).annotate(x=hl.rand_bool(0.5, seed=0))
>>> table.x.collect()  
[0.5488135008937808,
 0.9595974306263271,
 0.42205690070893265,
 0.828743805759555,
 0.6414977904324134]

The seed can also be set globally using set_global_seed(). This sets the seed globally for all subsequent Hail operations, and a pipeline will be guaranteed to have the same results if the global seed is set right beforehand:

>>> hl.set_global_seed(0)
>>> hl.eval(hl.array([hl.rand_unif(0, 1), hl.rand_unif(0, 1)]))  
[0.6830630912401323, 0.4035978197966855]

>>> hl.set_global_seed(0)
>>> hl.eval(hl.array([hl.rand_unif(0, 1), hl.rand_unif(0, 1)]))  
[0.6830630912401323, 0.4035978197966855]

rand_bool(p[, seed])

Returns True with probability p.

rand_beta(a, b[, lower, upper, seed])

Samples from a beta distribution with parameters a (alpha) and b (beta).

rand_cat(prob[, seed])

Samples from a categorical distribution.

rand_dirichlet(a[, seed])

Samples from a Dirichlet distribution.

rand_gamma(shape, scale[, seed])

Samples from a gamma distribution with parameters shape and scale.

rand_norm([mean, sd, seed])

Samples from a normal distribution with mean mean and standard deviation sd.

rand_pois(lamb[, seed])

Samples from a Poisson distribution with rate parameter lamb.

rand_unif([lower, upper, seed])

Samples from a uniform distribution within the interval [lower, upper].

rand_int32(a[, b, seed])

Samples from a uniform distribution of 32-bit integers.

rand_int64(a[, b, seed])

Samples from a uniform distribution of 64-bit integers.

shuffle(a[, seed])

Randomly permute an array
hail.expr.functions.rand_bool(p, seed=None)[source]

Returns True with probability p.

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_bool(0.5))
False

>>> hl.eval(hl.rand_bool(0.5))
True
Parameters
Returns

BooleanExpression

hail.expr.functions.rand_beta(a, b, lower=None, upper=None, seed=None)[source]

Samples from a beta distribution with parameters a (alpha) and b (beta).

Notes

The optional parameters lower and upper represent a truncated beta distribution with parameters a and b and support [lower, upper]. Draws are made via rejection sampling, i.e. returning the first draw from Beta(a,b) that falls in range [lower, upper]. This procedure may be slow if the probability mass of Beta(a,b) over [lower, upper] is small.

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_beta(0.5, 0.5))
0.3483677318466065

>>> hl.eval(hl.rand_beta(2, 5))
0.23894608018057753
Parameters
Returns

Float64Expression

hail.expr.functions.rand_cat(prob, seed=None)[source]

Samples from a categorical distribution.

Notes

The categories correspond to the indices of prob, an unnormalized probability mass function. The probability of drawing index i is prob[i]/sum(prob).

Warning

This function may be slow when the number of categories is large.

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_cat([0, 1.7, 2]))
2

>>> hl.eval(hl.rand_cat([0, 1.7, 2]))
1
Parameters
Returns

Int32Expression

hail.expr.functions.rand_dirichlet(a, seed=None)[source]

Samples from a Dirichlet distribution.

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_dirichlet([1, 1, 1]))
[0.31600799564679466, 0.22921566396520351, 0.45477634038800185]

>>> hl.eval(hl.rand_dirichlet([1, 1, 1]))
[0.28935842257116556, 0.40020478428981887, 0.31043679313901557]
Parameters
Returns

Float64Expression

hail.expr.functions.rand_gamma(shape, scale, seed=None)[source]

Samples from a gamma distribution with parameters shape and scale.

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_gamma(1, 1))
0.8934929450909811

>>> hl.eval(hl.rand_gamma(1, 1))
0.3423233699402248
Parameters
Returns

Float64Expression

hail.expr.functions.rand_norm(mean=0, sd=1, seed=None)[source]

Samples from a normal distribution with mean mean and standard deviation sd.

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_norm())
0.30971254606692267

>>> hl.eval(hl.rand_norm())
-1.6120679347033475
Parameters
Returns

Float64Expression

hail.expr.functions.rand_pois(lamb, seed=None)[source]

Samples from a Poisson distribution with rate parameter lamb.

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_pois(1))
1.0

>>> hl.eval(hl.rand_pois(1))
1.0
Parameters
Returns

Float64Expression

hail.expr.functions.rand_unif(lower=0.0, upper=1.0, seed=None)[source]

Samples from a uniform distribution within the interval [lower, upper].

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_unif())
0.6830630912401323

>>> hl.eval(hl.rand_unif(0, 1))
0.4035978197966855

>>> hl.eval(hl.rand_unif(0, 1))
0.26020045338162423
Parameters
Returns

Float64Expression

hail.expr.functions.rand_int32(a, b=None, *, seed=None)[source]

Samples from a uniform distribution of 32-bit integers.

If b is None, samples from the uniform distribution over [0, a). Otherwise, sample from the uniform distribution over [a, b).

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_int32(10))
0

>>> hl.eval(hl.rand_int32(10, 15))
11

>>> hl.eval(hl.rand_int32(10, 15))
12
Parameters

  • a (int or Int32Expression) – If b is None, the right boundary of the range; otherwise, the left boundary of range.

  • b (int or Int32Expression) – If specified, the right boundary of the range.

  • seed (int, optional) – Random seed.

Returns

Int32Expression

hail.expr.functions.rand_int64(a, b=None, *, seed=None)[source]

Samples from a uniform distribution of 64-bit integers.

If b is None, samples from the uniform distribution over [0, a). Otherwise, sample from the uniform distribution over [a, b).

Examples

>>> hl.set_global_seed(0)
>>> hl.eval(hl.rand_int64(10))
2

>>> hl.eval(hl.rand_int64(1 << 33, 1 << 35))
13313179445

>>> hl.eval(hl.rand_int64(1 << 33, 1 << 35))
18981019040
Parameters

  • a (int or Int64Expression) – If b is None, the right boundary of the range; otherwise, the left boundary of range.

  • b (int or Int64Expression) – If specified, the right boundary of the range.

  • seed (int, optional) – Random seed.

Returns

Int64Expression

hail.expr.functions.shuffle(a, seed=None)[source]

Randomly permute an array

Example

>>> hl.set_global_seed(0)
>>> hl.eval(hl.shuffle(hl.range(5)))
[3, 2, 0, 4, 1]
Parameters
Returns

ArrayExpression