nd¶

NDArray Functions

Notes¶

This is a recently added, experimental module. We would love to hear what use cases you have for this as we expand this functionality. As much as possible, we try to mimic the numpy array interface.

`array`(input_array[, dtype])	Construct an `NDArrayExpression`
`arange`(start[, stop, step])	Returns a 1-dimensions ndarray of integers from start to stop by step.
`full`(shape, value[, dtype])	Creates a hail `NDArrayNumericExpression` full of the specified value.
`zeros`(shape[, dtype])	Creates a hail `NDArrayNumericExpression` full of zeros.
`ones`(shape[, dtype])	Creates a hail `NDArrayNumericExpression` full of ones.
`diagonal`(nd)	Gets the diagonal of a 2 dimensional NDArray.
`solve`(a, b[, no_crash])	Solve a linear system.
`solve_triangular`(nd_coef, nd_dep[, lower])	Solve a triangular linear system.
`qr`(nd[, mode])	Performs a QR decomposition.
`svd`(nd[, full_matrices, compute_uv])	Performs a singular value decomposition.
`inv`(nd)	Performs a matrix inversion.
`concatenate`(nds[, axis])	Join a sequence of arrays along an existing axis.
`hstack`(arrs)	Stack arrays in sequence horizontally (column wise).
`vstack`(arrs)	Stack arrays in sequence vertically (row wise).
`eye`(N[, M, dtype])	Construct a 2-D `NDArrayExpression` with ones on the main diagonal and zeros elsewhere.
`identity`(N[, dtype])	Constructs a 2-D `NDArrayExpression` representing the identity array.
`maximum`(nd1, nd2)	Compares elements at corresponding indexes in arrays and returns an array of the maximum element found at each compared index.
`minimum`(nd1, nd2)	Compares elements at corresponding indexes in arrays and returns an array of the minimum element found at each compared index.

hail.nd.array(input_array, dtype=None)[source]¶

Construct an NDArrayExpression

Examples

>>> hl.eval(hl.nd.array([1, 2, 3, 4]))
array([1, 2, 3, 4], dtype=int32)

>>> hl.eval(hl.nd.array([[1, 2, 3], [4, 5, 6]]))
array([[1, 2, 3],
   [4, 5, 6]], dtype=int32)

>>> hl.eval(hl.nd.array(np.identity(3)))
array([[1., 0., 0.],
   [0., 1., 0.],
   [0., 0., 1.]])

>>> hl.eval(hl.nd.array(hl.range(10, 20)))
array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19], dtype=int32)

Parameters

input_array (ArrayExpression, numpy ndarray, or nested python lists/tuples) – The array to convert to a Hail ndarray.
dtype (HailType) – Desired hail type. Default: float64.

Returns

NDArrayExpression – An ndarray based on the input array.

hail.nd.arange(start, stop=None, step=1)[source]¶

Returns a 1-dimensions ndarray of integers from start to stop by step.

Examples

>>> hl.eval(hl.nd.arange(10))
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int32)

>>> hl.eval(hl.nd.arange(3, 10))
array([3, 4, 5, 6, 7, 8, 9], dtype=int32)

>>> hl.eval(hl.nd.arange(0, 10, step=3))
array([0, 3, 6, 9], dtype=int32)

Notes

The range includes start, but excludes stop.

If provided exactly one argument, the argument is interpreted as stop and start is set to zero. This matches the behavior of Python’s range.

Parameters

start (int or Expression of type tint32) – Start of range.
stop (int or Expression of type tint32) – End of range.
step (int or Expression of type tint32) – Step of range.

Returns

NDArrayNumericExpression – A 1-dimensional ndarray from start to stop by step.

hail.nd.full(shape, value, dtype=None)[source]¶

Creates a hail NDArrayNumericExpression full of the specified value.

Examples

Create a 5 by 7 NDArray of type tfloat64 9s.

>>> hl.nd.full((5, 7), 9)

It is possible to specify a type other than tfloat64 with the dtype argument.

>>> hl.nd.full((5, 7), 9, dtype=hl.tint32)

Parameters

shape (tuple or TupleExpression) – Desired shape.
value (Expression or python value) – Value to fill ndarray with.
dtype (HailType) – Desired hail type.

Returns

NDArrayNumericExpression – An ndarray of the specified shape filled with the specified value.

hail.nd.zeros(shape, dtype=dtype('float64'))[source]¶

Creates a hail NDArrayNumericExpression full of zeros.

Examples

Create a 5 by 7 NDArray of type tfloat64 zeros.

>>> hl.nd.zeros((5, 7))

It is possible to specify a type other than tfloat64 with the dtype argument.

>>> hl.nd.zeros((5, 7), dtype=hl.tfloat32)

Parameters

shape (tuple or TupleExpression) – Desired shape.
dtype (HailType) – Desired hail type. Default: float64.

See also

full()

Returns: NDArrayNumericExpression – ndarray of the specified size full of zeros.

hail.nd.ones(shape, dtype=dtype('float64'))[source]¶

Creates a hail NDArrayNumericExpression full of ones.

Examples

Create a 5 by 7 NDArray of type tfloat64 ones.

>>> hl.nd.ones((5, 7))

It is possible to specify a type other than tfloat64 with the dtype argument.

>>> hl.nd.ones((5, 7), dtype=hl.tfloat32)

Parameters

shape (tuple or TupleExpression) – Desired shape.
dtype (HailType) – Desired hail type. Default: float64.

See also

full()

Returns: NDArrayNumericExpression – ndarray of the specified size full of ones.

hail.nd.diagonal(nd)[source]¶

Gets the diagonal of a 2 dimensional NDArray.

Examples

>>> hl.eval(hl.nd.diagonal(hl.nd.array([[1, 2], [3, 4]])))
array([1, 4], dtype=int32)

Parameters: nd (NDArrayNumericExpression) – A 2 dimensional NDArray, shape(M, N).
Returns: NDArrayExpression – A 1 dimension NDArray of length min(M, N), containing the diagonal of nd.

hail.nd.solve(a, b, no_crash=False)[source]¶

Solve a linear system.

Parameters

a (NDArrayNumericExpression, (N, N)) – Coefficient matrix.
b (NDArrayNumericExpression, (N,) or (N, K)) – Dependent variables.

Returns

NDArrayNumericExpression, (N,) or (N, K) – Solution to the system Ax = B. Shape is same as shape of B.

hail.nd.solve_triangular(nd_coef, nd_dep, lower=False)[source]¶

Solve a triangular linear system.

Parameters

nd_coef (NDArrayNumericExpression, (N, N)) – Triangular coefficient matrix.
nd_dep (NDArrayNumericExpression, (N,) or (N, K)) – Dependent variables.
lower (bool:) – If true, nd_coef is interpreted as a lower triangular matrix If false, nd_coef is interpreted as a upper triangular matrix

Returns

NDArrayNumericExpression, (N,) or (N, K) – Solution to the triangular system Ax = B. Shape is same as shape of B.

hail.nd.qr(nd, mode='reduced')[source]¶

Performs a QR decomposition.

If K = min(M, N), then:

reduced: returns q and r with dimensions (M, K), (K, N)
complete: returns q and r with dimensions (M, M), (M, N)
r: returns only r with dimensions (K, N)
raw: returns h, tau with dimensions (N, M), (K,)

Notes

The reduced QR, the default output of this function, has the following properties:

\[m \gte n \\ nd : \mathbb{R}^{m \times n} \\ Q : \mathbb{R}^{m \times n} \\ R : \mathbb{R}^{n \times n} \\ \\ Q^T Q = \mathbb{1}\]

The complete QR, has the following properties:

\[m \gte n \\ nd : \mathbb{R}^{m \times n} \\ Q : \mathbb{R}^{m \times m} \\ R : \mathbb{R}^{m \times n} \\ \\ Q^T Q = \mathbb{1} Q Q^T = \mathbb{1}\]

Parameters

nd (NDArrayExpression) – A 2 dimensional ndarray, shape(M, N)
mode (str) – One of “reduced”, “complete”, “r”, or “raw”. Defaults to “reduced”.

Returns

- q (ndarray of float64) – A matrix with orthonormal columns.
- r (ndarray of float64) – The upper-triangular matrix R.
- (h, tau) (ndarrays of float64) – The array h contains the Householder reflectors that generate q along with r. The tau array contains scaling factors for the reflectors

hail.nd.svd(nd, full_matrices=True, compute_uv=True)[source]¶

Performs a singular value decomposition.

Parameters

nd (NDArrayNumericExpression) – A 2 dimensional ndarray, shape(M, N).
full_matrices (bool) – If True (default), u and vt have dimensions (M, M) and (N, N) respectively. Otherwise, they have dimensions (M, K) and (K, N), where K = min(M, N)
compute_uv (bool) – If True (default), compute the singular vectors u and v. Otherwise, only return a single ndarray, s.

Returns

- u (NDArrayNumericExpression) – The left singular vectors.
- s (NDArrayNumericExpression) – The singular values.
- vt (NDArrayNumericExpression) – The right singular vectors.

hail.nd.inv(nd)[source]¶

Performs a matrix inversion.

Parameters: nd (NDArrayNumericExpression) – A 2 dimensional ndarray.
Returns: NDArrayNumericExpression – The inverted matrix.

hail.nd.concatenate(nds, axis=0)[source]¶

Join a sequence of arrays along an existing axis.

Examples

>>> x = hl.nd.array([[1., 2.], [3., 4.]])
>>> y = hl.nd.array([[5.], [6.]])
>>> hl.eval(hl.nd.concatenate([x, y], axis=1))
array([[1., 2., 5.],
       [3., 4., 6.]])
>>> x = hl.nd.array([1., 2.])
>>> y = hl.nd.array([3., 4.])
>>> hl.eval(hl.nd.concatenate((x, y), axis=0))
array([1., 2., 3., 4.])

Parameters

nds (a sequence of NDArrayNumericExpression) – The arrays must have the same shape, except in the dimension corresponding to axis (the first, by default). Note: unlike Numpy, the numerical element type of each array_like must match.
axis (int, optional) – The axis along which the arrays will be joined. Default is 0. Note: unlike Numpy, if provided, axis cannot be None.

Returns

NDArrayExpression – The concatenated array

hail.nd.hstack(arrs)[source]¶

Stack arrays in sequence horizontally (column wise). Equivalent to concatenation along the second axis, except for 1-D arrays where it concatenates along the first axis.

This function makes most sense for arrays with up to 3 dimensions. concatenate() provides more general stacking and concatenation operations.

Parameters: tup (sequence of NDArrayExpression) – The arrays must have the same shape along all but the second axis, except 1-D arrays which can be any length.
Returns: NDArrayExpression – The array formed by stacking the given arrays.

See also

concatenate(), vstack()

Examples

>>> a = hl.nd.array([1,2,3])
>>> b = hl.nd.array([2, 3, 4])
>>> hl.eval(hl.nd.hstack((a,b)))
array([1, 2, 3, 2, 3, 4], dtype=int32)
>>> a = hl.nd.array([[1],[2],[3]])
>>> b = hl.nd.array([[2],[3],[4]])
>>> hl.eval(hl.nd.hstack((a,b)))
array([[1, 2],
       [2, 3],
       [3, 4]], dtype=int32)

hail.nd.vstack(arrs)[source]¶

Stack arrays in sequence vertically (row wise). 1-D arrays of shape (N,), will reshaped to (1,N) before concatenation. For all other arrays, equivalent to concatenate() with axis=0.

Examples

>>> a = hl.nd.array([1, 2, 3])
>>> b = hl.nd.array([2, 3, 4])
>>> hl.eval(hl.nd.vstack((a,b)))
array([[1, 2, 3],
       [2, 3, 4]], dtype=int32)
>>> a = hl.nd.array([[1], [2], [3]])
>>> b = hl.nd.array([[2], [3], [4]])
>>> hl.eval(hl.nd.vstack((a,b)))
array([[1],
       [2],
       [3],
       [2],
       [3],
       [4]], dtype=int32)

Parameters: arrs (sequence of NDArrayExpression) – The arrays must have the same shape along all but the first axis. 1-D arrays must have the same length.
Returns: NDArrayExpression – The array formed by stacking the given arrays, will be at least 2-D.

See also

concatenate(): Join a sequence of arrays along an existing axis.

hail.nd.eye(N, M=None, dtype=dtype('float64'))[source]¶

Construct a 2-D NDArrayExpression with ones on the main diagonal and zeros elsewhere.

Examples

>>> hl.eval(hl.nd.eye(3))
array([[1., 0., 0.],
       [0., 1., 0.],
       [0., 0., 1.]])
>>> hl.eval(hl.nd.eye(2, 5, dtype=hl.tint32))
array([[1, 0, 0, 0, 0],
       [0, 1, 0, 0, 0]], dtype=int32)

Parameters

N (NumericExpression or Python number) – Number of rows in the output.
M (NumericExpression or Python number, optional) – Number of columns in the output. If None, defaults to N.
dtype (numeric HailType, optional) – Element type of the returned array. Defaults to tfloat64

Returns

NDArrayExpression – An (N, M) matrix with ones on the main diagonal, zeros elsewhere.

See also

identity(), diagonal()

hail.nd.identity(N, dtype=dtype('float64'))[source]¶

Constructs a 2-D NDArrayExpression representing the identity array. The identity array is a square array with ones on the main diagonal.

Examples

>>> hl.eval(hl.nd.identity(3))
array([[1., 0., 0.],
       [0., 1., 0.],
       [0., 0., 1.]])

Parameters

n (NumericExpression or Python number) – Number of rows and columns in the output.
dtype (numeric HailType, optional) – Element type of the returned array. Defaults to tfloat64

Returns

NDArrayExpression – An (N, N) matrix with its main diagonal set to one, and all other elements 0.

See also

eye()

hail.nd.maximum(nd1, nd2)[source]¶

Compares elements at corresponding indexes in arrays and returns an array of the maximum element found at each compared index.

If an array element being compared has the value NaN, the maximum for that index will be NaN.

Examples

>>> a = hl.nd.array([1, 5, 3])
>>> b = hl.nd.array([2, 3, 4])
>>> hl.eval(hl.nd.maximum(a, b))
array([2, 5, 4], dtype=int32)
>>> a = hl.nd.array([hl.float64(float("NaN")), 5.0, 3.0])
>>> b = hl.nd.array([2.0, 3.0, hl.float64(float("NaN"))])
>>> hl.eval(hl.nd.maximum(a, b))
array([nan, 5., nan])

Parameters

nd1 (NDArrayExpression)
nd2 (class:.NDArrayExpression, .ArrayExpression, numpy ndarray, or nested python lists/tuples.) – Nd1 and nd2 must be the same shape or broadcastable into common shape. Nd1 and nd2 must have elements of comparable types

Returns

NDArrayExpression – Element-wise maximums of nd1 and nd2. If nd1 has the same shape as nd2, the resulting array will be of that shape. If nd1 and nd2 were broadcasted into a common shape, the resulting array will be of that shape

hail.nd.minimum(nd1, nd2)[source]¶

Compares elements at corresponding indexes in arrays and returns an array of the minimum element found at each compared index.

If an array element being compared has the value NaN, the minimum for that index will be NaN.

Examples

>>> a = hl.nd.array([1, 5, 3])
>>> b = hl.nd.array([2, 3, 4])
>>> hl.eval(hl.nd.minimum(a, b))
array([1, 3, 3], dtype=int32)
>>> a = hl.nd.array([hl.float64(float("NaN")), 5.0, 3.0])
>>> b = hl.nd.array([2.0, 3.0, hl.float64(float("NaN"))])
>>> hl.eval(hl.nd.minimum(a, b))
array([nan, 3., nan])

Parameters

nd1 (NDArrayExpression)
nd2 (class:.NDArrayExpression, .ArrayExpression, numpy ndarray, or nested python lists/tuples.) – nd1 and nd2 must be the same shape or broadcastable into common shape. Nd1 and nd2 must have elements of comparable types

Returns

min_array (NDArrayExpression) – Element-wise minimums of nd1 and nd2. If nd1 has the same shape as nd2, the resulting array will be of that shape. If nd1 and nd2 were broadcasted into a common shape, resulting array will be of that shape