Hail is an open-source, scalable framework for exploring and analyzing genomic data.
The Hail project began in Fall 2015 to empower the worldwide genetics community to harness the flood of genomes to discover the biology of human disease. Since then, Hail has expanded to enable analysis of large-scale datasets beyond the field of genomics.
Here are two examples of projects powered by Hail:
For genomics applications, Hail can:
Hail’s functionality is exposed through Python and backed by distributed algorithms built on top of Apache Spark to efficiently analyze gigabyte-scale data on a laptop or terabyte-scale data on a cluster.
Users can script pipelines or explore data interactively in Jupyter notebooks that combine Hail’s methods, PySpark’s scalable SQL and machine learning algorithms, and Python libraries like pandas’s scikit-learn and Matplotlib. Hail also provides a flexible domain language to express complex quality control and analysis pipelines with concise, readable code.
There are currently two versions of Hail:
0.1 (stable) and
0.2 beta (development). We recommend that new users install
0.2 beta, since this version is already radically improved from
0.1, the file format is stable, and the interface is nearly stable.
To get started using Hail
0.2 beta on your own data or on public data:
As we work toward a stable
0.2 release, additional improvements to the interface may require users to modify their pipelines when updating to the latest patch. All such breaking changes will be logged here.
There are many ways to get in touch with the Hail team if you need help using Hail, or if you would like to suggest improvements or features. We also love to hear from new users about how they are using Hail.
Hail uses a continuous deployment approach to software development, which means we frequently add new features. We update users about changes to Hail via the Discussion Forum. We recommend creating an account on the Discussion Forum so that you can subscribe to these updates.
Hail is committed to open-source development. Our Github repo is publicly visible. If you’d like to contribute to the development of methods or infrastructure, please:
The Hail team is embedded in the Neale lab at the Stanley Center for Psychiatric Research of the Broad Institute of MIT and Harvard and the Analytic and Translational Genetics Unit of Massachusetts General Hospital.
Contact the Hail team at
Follow Hail on Twitter @hailgenetics.
If you use Hail for published work, please cite the software:
Hail team uses IntelliJ and YourKit Java Profiler. We would like to thank YourKit, LLC for generously supporting open-source projects with free licenses.