This release is mainly to address build errors resulting from functionality introduced in 1.7.0, i.e., n-gram blocking on the GPU. Full release notes below:

General Fixes/Improvements

Protect loading of n-gram blocking on GPU (#4779)
General lint fixes (#4771)
Increase CI parallelism (#4702)
Update DialCrowd to mephisto 2.0.1

Agent Improvements/Fixes

[BB3] General fixes (#4786, #4789)
[BB3] Memory usage heuristics (#4770)
[BB3] README Updates (#4784)
[DIRECTOR] Added shared embedding option to director model. (#4763)

Assets 2

23 Aug 22:36

klshuster

1.7.0

a5e4685

1.7.0

New Releases

BlenderBot 3 is now available in ParlAI! (#4709, #4710, #4711, #4712, #4713, #4715, #4716, #4725, #4746, #4747, #4753).
- Includes the code from the paper, "Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback" (#4714)
- Includes the code from the paper, "Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls"
The DIRECTOR Project + Agent code is available in ParlAI (#4597, #4602, #4603, #4605, #4607, #4738, #4701)
[DialCrowd] Incorporate the DialCrowd crowdsourcing toolkit into ParlAI (#4387)

New Features

The Decoder-Only Transformer agent is now available in ParlAI! (#4329)
Beam N-Gram blocking is now supported on the GPU (#4633, #4721)
The model chat mephisto crowdsourced task now supports emojis! (#4666)

Agent Fixes + Improvements

Updates to Reranker, Pacer (#4469, #4473, #4488)
Updates to RagAgent, FidAgent, and Search (#4470, #4503, #4631, #4765)
Updates to HuggingFace agents (#4508)
Updates to T5Agent (#4505)
Updates to BertClassifierAgent (#4553)
Updates to SeeKeR (#4634)
Updates to TorchAgent, TorchGeneratorAgent, TorchClassifierAgent` (#4654, #4650, #4700, #4727, #4754, #4720)
Updates to models from the WhoAmI project (#4742)

Bug fixes + Performance Improvements

Script fixes script (#4054, #4482, #4496, #4572)
[TorchScript] fixes (#4489, #4499, #4554), add torchscriptable classifier + BPE tokenizer (#4566),
Train model supports setting seed (#4662)
ParlaiParser Fix (#4507)
Various crowdsourcing fixes + enhancements (#4551, #4560, #4577, #4625)
Fix downloads of TOD models (#4647)
Test fixes (#4749)

Datasets & Teachers & Mutators

Added datasets:
- SPOLIN Dataset (#4540)
- Friends Dataset (#4568, #4678, #4693)
Deprecate Auxiliary BST Teachers (#4513)
Updates to HuggingFace dataset integration (#4516)
Update LCCC Download Link (#4542)
MultiWoz V22 Updates (#4565, #4656, #4695, #4752, #4757, #4761, #4762, #4736, #4764)
Wizard of Internet Updates (#4576)
SaferDialogues updates (#4623)
Various mutators bugs (#4730)
BAD Teacher updates (#4732)
GoogleSGD updates (#4698)

Logging & Metrics

Improvements to Weights & Biases integration (#4484, #4548, #4708)
Include tensorboard logging in eval_model (#4497)
TimerMetric fixes (#4536)
Expose precision + recall metrics (#4670)

Developers & Documentation

Various documentation updates (#4487, #4509, #4609, #4613, #4612, #4624, #4632, #4687, #4688, #4690, #4691, #4686, #4733)
Various dependency + CI updates (#4481, #4506, #4514, #4483, #4524, #4523, #4522, #4521, #4529, #4532, #4562, #4564, #4590, #4608, #4593, #4615, #4671, #4677, #4759)
Various linting, syntactic updates (#4684, #4718, #4728, #4731)

Assets 2

30 Mar 19:30

klshuster

1.6.0

054a0ff

v1.6.0

New Releases

SeeKeR project code and models release (#4447, #4458, #4449, #4453, #4448)
User Simulators and Task Oriented Dialogue support (#4437, #4438, #4188, #4187, #4189, #4186, #4185, #4184, #4183, #4249, #4174, #4181, #4182, #4180, #4178, #4233)
Per Turn evaluation project code release (#4304, #4323, #4333, #4362)
SaFeRDialogues dataset and model release (#4229, #4300)
Am I Me or You project code release (#4239, #4252, #4250)
K2R Project release(#4251)

New Features

Updating to Mephisto 1.0 (#4426)
[TGA]
- More flexible token metadata logging (#4169, #4427)
- Record avg generation length (#4295)
- change tga default to not sort (#4138)
Add a small example Flask server (#4433)
Add support for WorldLoggers in training (#4369)
[T5] Support Distributed Training (#4434)
[HuggingFace] Add support for any GPT-2 model hosted in Huggingface (#4360), and ranking (#4326)
[Crowdsourcing]
- [ACUTE-Eval] Add support for knowledgeable question (#4416), interestingness question (#4113)
- [ACUTE-Eval] Record start and end times (#4208)
- [Model chat] allow multiple final ratings (#4276)
- [Model chat] allow spacing out of annotation buckets (#4275)
- Allow Use of external database in crowdsourcing code (#4272)
- Allow specification of blueprints from command line (#4254)
- Unify turn annotation tasks (model chat and turn annotations static tasks) (#4162)
- [ACUTE-Eval] Save worker name in Acute-Eval analysis script (#4126)
[RAG/FiD/BB2]
- [All] Incremental Decoding (#4088)
- [BB2] Allow hybrid mode (skip search when needed) (#4221)
- [FiD] Add specialized chunking to search engine retrievers (#4227)
- [BB2] Support for gold docs training (#4163)
- [FiD] Gold retrieved documents FiD Agent. (#4123)
- Export memories to the observation output (#4040)
[Style-Controlled Generation] OSS Second Classifier (#4380)
[Re-ranker] Support for a classifier re-ranker agent (#4291)
[TCA] return candidates (#4286)
Curated response generators (#4197)
[Chat Services] add option for host specification (#4335)

Bug fixes

[TCA] Various updates and fixes (#4287, #4406, #4270)
[BlenderBot2] Various updates and fixes (#4379, #4428, #4419, #4331, #4377, #4366, #4289, #4259, #4238, #4212, #4198, #4156)
[TorchAgent] Fix an issue with warmup updates not working properly (introduced after 1.5.1) (#4384, #4242, #4196)
[RAG/FiD] various updates and fixes (#4436, #4361, #4389, #4207, #4199, #4146)
[GPT2] misc. Fixes (#4395)
[Train Model] Fixes train_model worldlogging for multitask with mutators. (#4414)
[Crowdsourcing] various updates and fixes (#4413, #4273, #4274, #4089)
[Re-ranker] Bubble up all batch fields (#4296)
Fix clobbering issues with WandB & Tensorboard requeueing (#4175, #4093)
Fix interactive web (#4140)
Several misc. fixes (#4459, #4403, #4386, #4439, #4425, #4021, #4218, #4164)

Datasets & Teachers & Mutators

Relicense several parlai datasets as commercially friendly (#4269, #4213, #4126)
LLLC, a large chinese dataset (#4325)
Casino dataset (#4129)
Upgrade internals of several teachers: WoW (#4284), Empathetic Dialogues (#4405), Natural Questions (#4205)
WoW and WoI mutators (#4418, #4204, #4168, #4124, #4125, #4122, #4114)
Speed up the json teachers (#4404)
XPersona Dataset (#4314)
ConversationTeacher parent class is now ParlAIDialogTeacher (#4256)
[WizInt] Additional knowledge-related eval metrics (#4193); turn dicts to Messages (#4144)

Developers

Various documentation improvements (#4423, #4352, #4375, #4074, #4172, #4423, #4100, #4385. #4349, #4298, #4297, #4217, #4173, #4133, #4120, #4028, #4104, #4102)
Various test improvements and fixes (#4442, #4375, #4311, #4271, #4435, #4343, #4236, #4261, #4223, #4224, #4096, #4315, #4253, #4190)
Various dependency bumps (#4464, #4463, #4420, #4351, #4348, #4346, #4337, #4317, #4302, #4299, #4301)
Other QOL Improvements (#4095)

Assets 2

12 Oct 19:29

stephenroller

v1.5.1

f45fde0

v1.5.1

Project Releases

Hi, my name is Martha - New reduced-bias dialogue models! (#3981)

Minor features

Allow the style classifier to save output probabilities (#4026)
Add gpu option to torchscript BART models (#3979)

Crowdsourcing

Upgrade to Mephisto 0.4.0 (#3982, #4043)
Save model chat data in the Mephisto database (#4005)
Add Last Turn Annotation Only option to turn annotations static task (#3436)
Crowdsourcing data compiler class for using Mephisto abstractions (#4029, #4034)
Force model chats starting with "Hi!" to use BST-style context (#4004)

Datasets

[LIGHT] Jericho World dataset (#3957)

Bugfixes

[RAG] Fix ReGReT Cuda Issue (#4022)
[BlenderBot2] Handle distributed (#4023)
[mutators] Prevent name collisions in mutators (#4006)
[crowdsourcing] Fix model chat frame height by (#4030)

Other

Various error message improvements (#3987, #4018, #4007)
Various typos and documentation clarifications (#4053, #4052, #4008, #4002, #4064)
Various project-specific small additions and mutators (#3997, #3996, #3999, #4000, #3966, #4062)

Developer changes

[Modular] Update Transformer Layer __init__s by @klshuster in #4061
Migrate to the main branch by @stephenroller (#3998, #4010)
Allow more subclassing of self-chat world (#3955)
Moving Wizard of Internet task to crowdsourcing/projects directory. by @mojtaba-komeili in #3978
Improve internal facebook compatibility (#3964, #3994, #4032, #3991)
Various dependency bumps (#3995, #4072, #4017, #4077)

Contributors

stephenroller, klshuster, and mojtaba-komeili

Assets 2

25 Aug 13:23

stephenroller

v1.5.0

14bccc5

v1.5.0

Major Features

Model Cards have now been added to ParlAI. We support automatically generating cards for different models. See an example of BlenderBot2's model card. (#3857, #3865, #3899, #3884, #3915, #3965, #3860, #3863)

Minor Features

Add support for an extra final evaluation in training scripts with custom opts (#3883)
Add --checkpoint-activations to lower memory usage of training transformers (#3864)

Crowdsourcing

Open source the Wizard of Internet crowdsourcing tooling (#3924)
Open source the Personal Knowledge crowdsourcing tooling (#3945)
More customization for the [Static Turn Annotations] task. (#3926)
More flexibility in model-chat analysis code (#3844, #3958, #3935)

Bug Fixes

[world logs] Fix a bug where dynamic batching didn't use episode boundaries in world logs (#3867)
[self chat] Fix the order of openers so output is deterministic (#3923)
[parser] Fix bug where parse_kwargs couldn't handle mutator args (#3900)
[parser] Fix bug where nargs=+ wasn't working with parse_kwargs (#3930)
[teachers] Fixed an issue where some preprocessing was missing from labels field (#3874)
[blenderbot2] Fix a crash when not using internet search (#3950)
[regret] Fix a regression in Regret (#3934)
[rag] Fix a bug when receiving NaN scores

Teachers

Auxillary data added in the Wizard of Internet Task (#3897)

Documentation changes

Updates/improvements to some project documentations in Hallucination/RAG/etc (#3869, #3888, #3917, #3873)
Remove a false statement in chat services tutorial
Update a link to datasets in Contradiction (#3878)
Repo-wide spelling corrections (#3894, #3960)

Developer changes

Minor refactors (#3892)
CI fixes and improvements (#3954, #3837, #3889)
Small extensibilities to the torchscript functionality (#3851)
Enable self_chat to seed messages from tasks in parlai_internal (#3852)
Avoid exception in core/agents.py when arg is missing from dict (#3893)
Allow customization of the AcceptabilityChecker (#3846)

Assets 2

23 Jul 14:14

klshuster

1.4.1

5b71567

v1.4.1

Includes a fix to make sure that BlenderBot2 can be used with a pip install (#3848)

Also includes a teacher for training a Wikipedia Page Title Generator from the Wizard of Wikipedia Dataset (#3845)

Assets 2

22 Jul 19:19

klshuster

1.4.0

60f8235

v1.4.0

v1.4.0 Changelog

Major Features/Paper Releases

BlenderBot 2.0 Models, Code, Datasets

This release of ParlAI includes all the code, models, and datasets used to build BlenderBot 2.0 - please see the project page for more information, including how to access and use the models themselves (#3790, #3793, #3794, #3795, #3796, #3797, #3798, #3801, #3802, #3805, #3803, #3815, #3817)

Internet-Augmented Dialogue Generation

Build and release a crowd-sourced dataset of humans searching the internet and talking in depth about a vast array of topics; search-engine-augmented dialogue models are trained on the dataset, and are shown to be more knowledgeable than their non-search-augmented counterparts. (#3792, #3800, #3814)

Multi-Session Chat

Build and release a dataset of multi-session chats for the purpose of studying long-term open-domain conversation. Models trained on the dataset prove to perform better at recalling information from the distant past in the chats. (#3791, #3799, #3814)

Safety Benchmark Tests

With the release of Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling, ParlAI now supports running safety unit tests for conversational models. Check out the project page to see how to run the safety benchmarks. (#3767, #3768, #3769, #3784)

Convenience Functions for Agents

With a recent update to ParlAI's agent API, you can now bypass the act/observe loop for interacting with agents. (#3775)

In [1]: from parlai.core.agents import create_agent

In [2]: from parlai.core.params import ParlaiParser

In [3]: opt = ParlaiParser(True, True).parse_args(['--model_file', 'zoo:blender/blender_90M/model'])

In [4]: agent = create_agent(opt)

In [5]: agent.respond("Hi! How's it going?")
Out[5]: "it ' s going well . i ' m getting ready to go to work . how about you ?"

Minor Features

[Metrics] AUC Metrics added to parlai eval_model (#3751)

Bug Fixes

[Crowdsourcing] Crowdsourcing Fixes concerning Left Pane Text and Model Chat (#3773, #3780, #3789, #3831)
[RAG] Fix some issues with RAG/FiD agents (#3787, #3818, #3834, #3833, #3836)
Fix JSON serialization in interactive_web.py script (#3821)
Fix CI test issues (#3825, #3823)
Misc. README typos (#3807, #3839, #3841)

Developer Changes

Stripping before processing Conversations to allow leniency (#3772)
[ACUTE-Eval] Dev improvements to running ACUTE-Eval (#3781, #3782, #3783)
Allow self-chat in nested folders (#3785)
Allow opt presets from parlai_internal folders (#3819)

Assets 2

07 Jul 19:56

stephenroller

v1.3.0

bd9ac8f

v1.3.0

v1.3.0 Changelog

Major Features

Fully Sharded Data Parallel

Implementation of DeepSpeed/FairScale's Zero2 sharding. Improves training speed and reduces memory usage over vanilla DistributedDataParallel. Switch to the new mode with --ddp-backend zero2 to see free improvements in your training! (#3740)

Swappable Transformer Components

We've added support for overriding internal components within Transformers. It is now easy to swap only an attention module, or customize your layers, without having to fully override all classes. (#3567, #3703, #3708, #3715, #3638)

ChunkTeacher no longer requires num_examples/num_episodes to be correct

For as long as we've had ChunkTeacher, the value of num_examples/num_episodes must be exactly correct, or your training would hang. Furthermore, that calculation would need to be done outside of ParlAI. We've relaxed this restriction: these methods can now return arbitrary values, and you will correctly iterate through all of your data. However, using the wrong value of num_examples can cause the "epoch" counter (used in parlai train) to be wrong relative to your dataset. (#3681, #3745)

Eliminate dummy batches and init_cuda_buffer

You are no longer required to implement dummy batches in your Generator agents, when using custom batch formats. Additionally, you will no longer see a dummy batch as the first batch when debugging. Instead, the first batch your agent sees will be reserved as the future dummy batch. (#3732, #3744)

Paper Releases

Reducing Hallucination

Exploratory architectures that add retrieval mechanisms to dialogue models, reducing hallucination while maintaining conversational ability. (#3611, #3657, #3693, #3688, #3668)

Hash Layers & Ladder Transformers

More Parameters or More Compute? Answer: Both! Two new methods that explore this question: Hash Layers for more parameters, and Staircase Attention for more power per parameter. (#3697, #3699, #3700, #3746, #3747)

Minor Features

[TGA] Substantial speedups during generation on GPUs (#3730, #3729, #3669)
[Datasets] Add GLUE teachers, and support for HuggingFace datasets (#3570, #3624)
[Datasets] [Safety] Release the Non Adversarial Data (#3684)
[TA] Support temp history via special field in observation (#3617)
[TGA] Allow setting prefix tokens (#3760)
[TCA] Classifier on generator for TGA (#3716)
[ChunkTeacher] Remove exception for specifying non-streaming data (#3653)
[Transformer] Better initiaization for segment embeddings (#3680)
[Message] Add a new json_safe_payload method for serialization (#3643, #3726, #3686)
[JIT] Support special tokens in torchscript module. (#3644)
[JIT] Fix a parsing error with parlai torchscript in Python 3.8 (#3641)
[ACUTE] Support randomize_conversations (#3636, #3642)

Bugfixes

[train] Fix bugs with loading validation impatience. (#3713)
[train] Fix LR scheduler cooldown (#3719)
[train] Dynamic Batching doesn't choke with really small datasets (#3721)
[Logging] Fix a bug with world logging and multitasking (#3718)
[Mutators] Ensure mutations do not persist across epochs (#3649)
[BART] Do not add start/end tokens multiple times (#3714)
[TCA] weighted_f1 no longer assumes binary classification (#3728)
[Safety] Fix a Static Task bug and Safety README (#3612)
[logging] Fix an issue where --loglevel debug was ignored (#3658)
[Tensorboard] Fix an exception in some versions of Tensorboard (#3637)
[vacuum] Add support for PathManager in vacuum (#3635)
[Crowdsourcing] Slightly improve the analysis script to make it more robust (#3683, #3629)
Various locations where the change to is_padding caused issues (#3704, #3634, #3674)
Various typos/lint (#3621, #3622, #3646)

Developer changes

Helper functions for building deterministic data splits (#3676)
Teacher URL updates (#3749, #3627, #3678)
CI bugfixes & version bumps (#3754, #3724, #3672, #3652, #3710, #3628, #3452, #3720)
Documentation updates (#3748, #3690, #3742, #3671)
Mutators and Scripts support for parlai_internal (#3623, #3625)
[Crowdsourcing] Small refactor in Model-Chat

Assets 2

23 Apr 13:45

stephenroller

v1.2.0

6695ae7

v1.2.0

This Saturday marks the 4 year anniversary since the initial release of ParlAI. I'd like to offer my sincere gratitude to our users, our contributors, and all of the core development team. ParlAI wouldn't be what it is without all of you. -@stephenroller

Major new features

Background Preprocessing

Improve your training speeds by 1.25x-5.0x by switching from --num-workers 0 to --num-workers N. See our Speeding up training docs for details. (#3527, #3586, #3575, #3533, #3389)

(Beta) Support for torch.jit
Deploy faster models by exporting models with TorchScript. Currently limited to BART models only. (#3459)

Support for T5
We now have agents for Google's T5 models (#3519)

Opt Presets
Opt presets. Easily use prepackaged opt files as shorthand for long command line arguments (#3564)

Log/validate/stop based on number of steps
Get up to a 10% speedup of distributed training by switching from -vtim or -veps to -vstep (#3379, #3555)

Backwards-incompatible changes

DictionaryAgent.text2vec now requires input to be a string (#3472)
A number of older projects have been archived: DrQA (#3559), Controllable Dialogue (#3557), and Self-Feeding Chatbot (#3557).

Minor improvements

Performance speedup in generation using Transformer Generators (#3550)
Improvements to the Transformer API, making Transformer models more easily extensible. More to come soon. (#3486, #3545, #3466, #3501)
Various performance improvements when loading ParlAI or performing some activities (#3544, #3482)
Metrics:
- New truncation metrics show you how much context/label you're losing (#3458, #3508)
- Additional metrics in the Wizard of Wikipedia teacher (#3566, #3592, #3503)
- New token_em metric, an equivalent to accuracy with --skip-generation true (#3497)
Self-chat can now use seed messages (#3580)
New "normalized" ConvAI2 teachers for the non-standard variants (#3509)
Update FusedAdam support to use FairScale (#3522)
Add --wandb-entity flag to the logger (#3562)
Tensorboard now provides nicer names of metrics (#3534)

Bugfixes

[core] Fix a bug when resuming with the cosine LR scheduler after preemption (#3599)
[core] Improve robustness to serialization of Observations (#3591)
[core] ParlaiDialogTeacher now parses the rewards field as a number (#3517)
[core] Fix recently introduced ChunkTeacher bugs (#3549, #3542, #3543)
[core] Minor FP16 fixes when converting old checkpoints (#3514)
[core] Fix annoying ambiguity issues with commandline parsing in Python 3.8 (#3598)
[core] Fix a rare situation in case a dictionary contained tokens with leading whitespace (#3613)
[mutators] Fix a bug with the flatten mutator providing the wrong history (#3578, #3584)
[metrics] Fix a bug with computation of fairseq metrics (#3518)
[task] Fix a bug with Wizard of Wikipedia teacher causing some data to be omitted (#3585)
[task] Fix a crash in Wizard of Wikipedia end2end agent when given zero documents (#3602)
[task] Update a dead dataset to a new link (#3520)
[task] Fix an issue with CCPE (#3487)
[tga] Fix a case where TGA used as a ranking model could crash (#3541)
[agent] Fix a crash in BertDictionaryAgent (#3560)
[other] Various rare issues (#3505) and quality improvements (#3496)

Crowdsourcing improvements

Add new option to avoid randomizing pairs in ACUTE-Eval (#3528)
ACUTE-Eval provides additional warnings when options are not set to recommended values (#3536, #3524)

New Datasets

MetaLWoZ (#3583)
CMU_DoG (#3593, #3615)

Doc improvements

New tutorial: How to write a PR contributing (moving to a fork) tutorial (#3490)
New reference manual of standard ParlAI metrics (#3498)
Fix a bug keeping some tasks from being listed in our docs (#3495)
Various minor doc changes (#3513, #3561, #3596)

Developer improvements

Update to pytorch 1.8. No changes necessary. (#3607)
Various reliability improvements and speed ups to our CI and tests (#3603, #3597, #3604, #3606, #3605, #3587, #3588, #3590, #3539, #3535, #3526, #3521)
Various dependency bumps (#3553, #3537, #3515)
Switched logging.warn to logging.warning (#3569)
TorchAgent.history prettier debug printing (#3510)
Small other improvements (#3506)

Assets 2

0 Join discussion

Releases: facebookresearch/ParlAI

1.7.2

New Releases

New Features

Metrics

Bug fixes

Datasets & Teachers

Developers & Documentation

Repository maintenance

v1.7.1

General Fixes/Improvements

Agent Improvements/Fixes

1.7.0

New Releases

New Features

Agent Fixes + Improvements

Bug fixes + Performance Improvements

Datasets & Teachers & Mutators

Logging & Metrics

Developers & Documentation

v1.6.0

New Releases

New Features

Bug fixes

Datasets & Teachers & Mutators

Developers

v1.5.1

v1.5.1

Project Releases

Minor features

Crowdsourcing

Datasets

Bugfixes

Other

Developer changes

Contributors

v1.5.0

v1.5.0

Major Features

Minor Features

Crowdsourcing

Bug Fixes

Teachers

Documentation changes

Developer changes

v1.4.1

v1.4.0

v1.4.0 Changelog

Major Features/Paper Releases

BlenderBot 2.0 Models, Code, Datasets

Internet-Augmented Dialogue Generation

Multi-Session Chat

Safety Benchmark Tests

Convenience Functions for Agents

Minor Features

Bug Fixes

Developer Changes

v1.3.0

v1.3.0 Changelog

Major Features

Fully Sharded Data Parallel

Swappable Transformer Components

ChunkTeacher no longer requires num_examples/num_episodes to be correct

Eliminate dummy batches and init_cuda_buffer

Paper Releases

Reducing Hallucination

Hash Layers & Ladder Transformers

Minor Features

Bugfixes

Developer changes

v1.2.0

Major new features

Backwards-incompatible changes

Minor improvements

Bugfixes

Crowdsourcing improvements

New Datasets

Doc improvements

Developer improvements