Home » Catalog » Models » BERT Pretraining

BERT Pretraining

BERT Pretraining for PyTorch

Application

Natural Language Processing

Framework

PyTorch

GitHub Link

https://github.com/HabanaAI/Model-References/tree/master/PyTorch/nlp/bert

Pretrained Models

Checkpoints

SynapseAI 1.9.0

https://vault.habana.ai/artifactory/pretrained-models/checkpoints/1.9.0/PT/BertL-PT/BertL-PT-PyTorch-1.13.1-1.9.0-580-32n-checkpoint.tar.gz

Saved Models

SynapseAI 1.9.0

https://vault.habana.ai/artifactory/pretrained-models/models/1.9.0/PT/Bert-Large/Bert-Large-PyTorch-1.13.1-1.9.0-580-32n-model.tar.gz

Datasets

Wikipedia, Book Corpus

License

CC-BY-SA

Tags

BERT, Gaudi, Gaudi2, Natural Language Processing, NLP, Pretrained Model, PyTorch

Overview

Sign up for the latest Habana developer news, events, training, and updates.