Noah Constant

Generating author description...

All published works

Action	Title	Year	Authors
+ PDF Chat	Training LLMs over Neurally Compressed Text	2024	Brian Lester Jaehoon Lee Alex Alemi Jeffrey Pennington Adam Roberts Jascha Sohl‐Dickstein Noah Constant
+ PDF Chat	Transfer Learning for Text Diffusion Models	2024	Kehang Han Kathleen Kenealy Aditya Barua Noah Fiedel Noah Constant
+	UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining	2023	Hyung Won Chung Noah Constant Xavier García Adam Roberts Yi Tay Sharan Narang Orhan Fırat
+ PDF Chat	FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation	2023	Parker Riley Timothy Dozat Jan A. Botha Xavier García Dan Garrette Jason Riesa Orhan Fırat Noah Constant
+	Character-Aware Models Improve Visual Text Rendering	2023	Rosanne Liu Dan Garrette Chitwan Saharia William Chan Adam Roberts Sharan Narang Irina Blok Rj Mical Mohammad Norouzi Noah Constant
+	FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation	2023	Tu Vu Mohit Iyyer Xuezhi Wang Noah Constant Jerry Wei Jason Lee Chris Tar Yun-Hsuan Sung Denny Zhou Quoc V. Le
+	Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?	2023	C. Daniel Freeman Laura Culp Aaron Parisi Maxwell L. Bileschi Gamaleldin F. Elsayed Alex Rizkowsky Isabelle Simpson Alex Alemi Azade Nova Ben Adlam
+	Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models	2023	Avi Singh John D. Co-Reyes Rishabh Agarwal Ankesh Anand Piyush Patil Peter J. Liu J. Harrison Jaehoon Lee Kelvin Xu Aaron Parisi
+ PDF Chat	ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models	2022	Linting Xue Aditya Barua Noah Constant Rami Al‐Rfou Sharan Narang Mihir Kale Adam P. Roberts Colin Raffel
+ PDF Chat	Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models	2022	Jianmo Ni Gustavo Hernández Ábrego Noah Constant Ji Ma Keith Hall Daniel Cer Yinfei Yang
+ PDF Chat	SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer	2022	Tu Vu Brian Lester Noah Constant Rami Al‐Rfou Daniel Cer
+	Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation	2022	Tu Vu Aditya Barua Brian Lester Daniel Cer Mohit Iyyer Noah Constant
+	Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models	2022	Aarohi Srivastava Abhinav Rastogi Abhishek S. Rao Abu Awal Shoeb Abubakar Abid Adam Fisch Adam R. Brown Adam Santoro Aditya Gupta Adrià Garriga-Alonso
+	Reducing Retraining by Recycling Parameter-Efficient Prompts	2022	Brian Lester Joshua Yurtsever Siamak Shakeri Noah Constant
+	Bidirectional Language Models Are Also Few-shot Learners	2022	Ajay Patel Bryan Li Mohammad Sadegh Rasooli Noah Constant Colin Raffel Chris Callison-Burch
+	FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation	2022	Parker Riley Timothy Dozat Jan A. Botha Xavier García Dan Garrette Jason Riesa Orhan Fırat Noah Constant
+	Character-Aware Models Improve Visual Text Rendering	2022	Rosanne Liu Dan Garrette Chitwan Saharia William Chan Adam Roberts Sharan Narang Irina Blok Rj Mical Mohammad Norouzi Noah Constant
+ PDF Chat	Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation	2022	Tu Vu Aditya Barua Brian Lester Daniel Cer Mohit Iyyer Noah Constant
+	Towards Universality in Multilingual Text Rewriting.	2021	Xavier García Noah Constant Mandy Guo Orhan Fırat
+	TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling	2021	Parker Riley Noah Constant Mandy Guo Girish Kumar David Uthus Zarana Parekh
+	The Power of Scale for Parameter-Efficient Prompt Tuning	2021	Brian Lester Rami Al‐Rfou Noah Constant
+	The Power of Scale for Parameter-Efficient Prompt Tuning	2021	Brian Lester Rami Al‐Rfou Noah Constant
+	XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation	2021	Sebastian Ruder Noah Constant Jan A. Botha Aditya Siddhant Orhan Fırat Jinlan Fu Pengfei Liu Junjie Hu Graham Neubig Melvin Johnson
+	Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution	2021	Xavier García Noah Constant Ankur P. Parikh Orhan Fırat
+ PDF Chat	Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution	2021	Xavier García Noah Constant Ankur P. Parikh Orhan Fırat
+	nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?	2021	Mihir Kale Aditya Siddhant Noah Constant Melvin Johnson Rami Al‐Rfou Linting Xue
+ PDF Chat	mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer	2021	Linting Xue Noah Constant Adam P. Roberts Mihir Kale Rami Al‐Rfou Aditya Siddhant Aditya Barua Colin Raffel
+	Towards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension	2021	Siamak Shakeri Noah Constant Mihir Kale Linting Xue
+	nmT5 - Is parallel data still relevant for pre-training massively multilingual language models?	2021	Mihir Kale Aditya Siddhant Rami Al‐Rfou Linting Xue Noah Constant Melvin Johnson
+	TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling	2021	Parker Riley Noah Constant Mandy Guo Girish Kumar David Uthus Zarana Parekh
+ PDF Chat	XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation	2021	Sebastian Ruder Noah Constant Jan A. Botha Aditya Siddhant Orhan Fırat Jinlan Fu Pengfei Liu Junjie Hu Dan Garrette Graham Neubig
+ PDF Chat	The Power of Scale for Parameter-Efficient Prompt Tuning	2021	Brian Lester Rami Al‐Rfou Noah Constant
+	SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer	2021	Tu Vu Brian Lester Noah Constant Rami Al‐Rfou Daniel Cer
+	Towards Universality in Multilingual Text Rewriting	2021	Xavier García Noah Constant Mandy Guo Orhan Fırat
+	The Power of Scale for Parameter-Efficient Prompt Tuning	2021	Brian Lester Rami Al‐Rfou Noah Constant
+	XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation	2021	Sebastian Ruder Noah Constant Jan L. Botha Aditya Siddhant Orhan Fırat Jinlan Fu Pengfei Liu Junjie Hu Dan Garrette Graham Neubig
+	Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution	2021	Xavier García Noah Constant Ankur P. Parikh Orhan Fırat
+	ByT5: Towards a token-free future with pre-trained byte-to-byte models	2021	Linting Xue Aditya Barua Noah Constant Rami Al‐Rfou Sharan Narang Mihir Kale Adam Roberts Colin Raffel
+	Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models	2021	Jianmo Ni Gustavo Hernández Ábrego Noah Constant Ji Ma Keith Hall Daniel Cer Yinfei Yang
+	Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension.	2020	Siamak Shakeri Noah Constant Mihir Kale Linting Xue
+	TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling	2020	Parker Riley Noah Constant Mandy Guo Girish Kumar David Uthus Zarana Parekh
+	MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models	2020	Mandy Guo Yinfei Yang Daniel Cer Qinlan Shen Noah Constant
+	LAReQA: Language-agnostic answer retrieval from a multilingual pool	2020	Uma Roy Noah Constant Rami Al‐Rfou Aditya Barua Aaron B. Phillips Yinfei Yang
+	MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models	2020	Mandy Guo Yinfei Yang Daniel Cer Qinlan Shen Noah Constant
+	Multilingual Universal Sentence Encoder for Semantic Retrieval	2020	Yinfei Yang Daniel Cer Amin Ahmad Mandy Guo Jax Law Noah Constant Gustavo Hernández Ábrego Steve Yuan Chris Tar Yun-Hsuan Sung
+	mT5: A massively multilingual pre-trained text-to-text transformer	2020	Linting Xue Noah Constant Adam P. Roberts Mihir Kale Rami Al‐Rfou Aditya Siddhant Aditya Barua Colin Raffel
+ PDF Chat	LAReQA: Language-Agnostic Answer Retrieval from a Multilingual Pool	2020	Uma Roy Noah Constant Rami Al‐Rfou Aditya Barua Aaron B. Phillips Yinfei Yang
+	TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling	2020	Parker Riley Noah Constant Mandy Guo Girish Kumar David Uthus Zarana Parekh
+ PDF Chat	Character-Level Language Modeling with Deeper Self-Attention	2019	Rami Al‐Rfou Dokook Choe Noah Constant Mandy Guo Llion Jones
+	Multilingual Universal Sentence Encoder for Semantic Retrieval	2019	Yinfei Yang Daniel Cer Amin Ahmad Mandy Guo Jax Law Noah Constant Gustavo Hernández Ábrego Steve Yuan Chris Tar Yun-Hsuan Sung
+	Bridging the Gap for Tokenizer-Free Language Models	2019	Dokook Choe Rami Al‐Rfou Mandy Guo Heeyoung Lee Noah Constant
+	ReQA: An Evaluation for End-to-End Answer Retrieval Models	2019	Amin Ahmad Noah Constant Yinfei Yang Daniel Cer
+	Character-Level Language Modeling with Deeper Self-Attention	2018	Rami Al‐Rfou Dokook Choe Noah Constant Mandy Guo Llion Jones
+	Universal Sentence Encoder	2018	Daniel Cer Yinfei Yang Sheng-yi Kong Nan Hua Nicole Limtiaco Rhomni St. John Noah Constant Mario Guajardo-Céspedes Steve Yuan Chris Tar
+	Learning Semantic Textual Similarity from Conversations	2018	Yinfei Yang Steve Yuan Daniel Cer Sheng-yi Kong Noah Constant Petr Pilar Heming Ge Yun-Hsuan Sung Brian Strope Ray Kurzweil
+	Effective Parallel Corpus Mining using Bilingual Sentence Embeddings	2018	Mandy Guo Qinlan Shen Yinfei Yang Heming Ge Daniel Cer Gustavo Hernández Ábrego Keith Stevens Noah Constant Yun-Hsuan Sung Brian Strope
+	Effective Parallel Corpus Mining using Bilingual Sentence Embeddings	2018	Mandy Guo Qinlan Shen Yinfei Yang Heming Ge Daniel Cer Gustavo Hernández Ábrego Keith Stevens Noah Constant Yun-Hsuan Sung Brian Strope
+	Learning Semantic Textual Similarity from Conversations	2018	Yinfei Yang Steve Yuan Daniel Cer Sheng-yi Kong Noah Constant Petr Pilar Heming Ge Yun-Hsuan Sung Brian Strope Ray Kurzweil
+	Character-Level Language Modeling with Deeper Self-Attention	2018	Rami Al‐Rfou Dokook Choe Noah Constant Mandy Guo Llion Jones

Common Coauthors

Coauthor	Papers Together
Rami Al‐Rfou	18
Daniel Cer	16
Mandy Guo	16
Yinfei Yang	13
Orhan Fırat	11
Brian Lester	10
Aditya Barua	9
Mihir Kale	9
Yun-Hsuan Sung	8
Linting Xue	8
Ray Kurzweil	7
Dan Garrette	7
Aditya Siddhant	7
Brian Strope	7
Parker Riley	6
Colin Raffel	6
Gustavo Hernández Ábrego	6
Steve Yuan	5
Adam Roberts	5
Xavier García	5
Tu Vu	5
Rosanne Liu	5
Melvin Johnson	5
Sharan Narang	5
Heming Ge	4
Qinlan Shen	4
Jaehoon Lee	4
Siamak Shakeri	4
Chris Tar	4
Jan A. Botha	4
Dokook Choe	4
David Uthus	4
Zarana Parekh	4
Jascha Sohl‐Dickstein	4
Noah Fiedel	4
Roman Novak	3
Sheng-yi Kong	3
Girish Kumar	3
Jinlan Fu	3
Pengfei Liu	3
Jeffrey Pennington	3
Ankur P. Parikh	3
Sebastian Ruder	3
Mohit Iyyer	3
Junjie Hu	3
Xavier García	3
Amin Ahmad	3
Llion Jones	3
Graham Neubig	3
Adam P. Roberts	3

Commonly Cited References

Action	Title	Year	Authors	# of times referenced
+ PDF Chat	SQuAD: 100,000+ Questions for Machine Comprehension of Text	2016	Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang	18
+	Attention is All you Need	2017	Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser Illia Polosukhin	17
+	SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing	2018	Taku Kudo John T. E. Richardson	12
+	End-to-End Retrieval in Continuous Space	2018	Daniel Gillick Alessandro Presta Gaurav Singh Tomar	9
+	On the Cross-lingual Transferability of Monolingual Representations	2020	Mikel Artetxe Sebastian Ruder Dani Yogatama	9
+	Unsupervised Cross-lingual Representation Learning at Scale	2020	Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Édouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov	8
+	Attention Is All You Need	2017	Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser Illia Polosukhin	8
+ PDF Chat	XNLI: Evaluating Cross-lingual Sentence Representations	2018	Alexis Conneau Ruty Rinott Guillaume Lample Adina Williams Samuel Bowman Holger Schwenk Veselin Stoyanov	8
+ PDF Chat	mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer	2021	Linting Xue Noah Constant Adam P. Roberts Mihir Kale Rami Al‐Rfou Aditya Siddhant Aditya Barua Colin Raffel	8
+	Multilingual Denoising Pre-training for Neural Machine Translation	2020	Yinhan Liu Jiatao Gu Naman Goyal Xian Li Sergey Edunov Marjan Ghazvininejad Mike Lewis Luke Zettlemoyer	7
+ PDF Chat	Universal Language Model Fine-tuning for Text Classification	2018	Jeremy Howard Sebastian Ruder	7
+	MLQA: Evaluating Cross-lingual Extractive Question Answering	2020	Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk	7
+	A Call for Clarity in Reporting BLEU Scores	2018	Matt Post	7
+	XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization	2020	Junjie Hu Sebastian Ruder Aditya Siddhant Graham Neubig Orhan Fırat Melvin Johnson	6
+	RoBERTa: A Robustly Optimized BERT Pretraining Approach	2019	Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer Veselin Stoyanov	6
+	Efficient Natural Language Response Suggestion for Smart Reply	2017	Matthew Henderson Rami Al‐Rfou Brian Strope Yun-Hsuan Sung Lukács László Ruiqi Guo Sanjiv Kumar Miklós Bálint Ray Kurzweil	6
+	MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer	2020	Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder	6
+	VECO: Variable Encoder-decoder Pre-training for Cross-lingual Understanding and Generation	2021	Fuli Luo Wei Wang Jiahao Liu Yijia Liu Bin Bi Songfang Huang Fei Huang Luo Si	6
+	PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification	2019	Yinfei Yang Yuan Zhang Chris Tar Jason Baldridge	6
+	Deep Contextualized Word Representations	2018	Matthew E. Peters Mark E Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer	6
+	MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension	2019	Adam Fisch Alon Talmor Robin Jia Minjoon Seo Eunsol Choi Danqi Chen	6
+	A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference	2018	Adina Williams Nikita Nangia Samuel Bowman	6
+ PDF Chat	Supervised Learning of Universal Sentence Representations from Natural Language Inference Data	2017	Alexis Conneau Douwe Kiela Holger Schwenk Loïc Barrault Antoine Bordes	6
+	Prefix-Tuning: Optimizing Continuous Prompts for Generation	2021	Xiang Lisa Li Percy Liang	5
+	Parameter-Efficient Transfer Learning for NLP	2019	Neil Houlsby Andrei Giurgiu Stanisław Jastrzȩbski Bruna Morrone Quentin de Laroussilhe Andréa Gesmundo Mona Attariyan Sylvain Gelly	5
+	Cross-lingual Language Model Pretraining	2019	Guillaume Lample Alexis Conneau	5
+	ReQA: An Evaluation for End-to-End Answer Retrieval Models	2019	Amin Ahmad Noah Constant Yinfei Yang Daniel Cer	5
+	Multilingual Universal Sentence Encoder for Semantic Retrieval	2020	Yinfei Yang Daniel Cer Amin Ahmad Mandy Guo Jax Law Noah Constant Gustavo Hernández Ábrego Steve Yuan Chris Tar Yun-Hsuan Sung	5
+ PDF Chat	Neural Machine Translation of Rare Words with Subword Units	2016	Rico Sennrich Barry Haddow Alexandra Birch	5
+	Learning Semantic Textual Similarity from Conversations	2018	Yinfei Yang Steve Yuan Daniel Cer Sheng-yi Kong Noah Constant Petr Pilar Heming Ge Yun-Hsuan Sung Brian Strope Ray Kurzweil	5
+ PDF Chat	SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation	2017	Daniel Cer Mona Diab Eneko Agirre Iñigo López-Gazpio Lucia Specia	5
+	A large annotated corpus for learning natural language inference	2015	Samuel R. Bowman Gabor Angeli Christopher Potts Christopher D. Manning	5
+ PDF Chat	Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond	2019	Mikel Artetxe Holger Schwenk	4
+	MASS: Masked Sequence to Sequence Pre-training for Language Generation	2019	Kaitao Song Xu Tan Tao Qin Jianfeng Lu Tie‐Yan Liu	4
+ PDF Chat	Learning How to Ask: Querying LMs with Mixtures of Soft Prompts	2021	Guanghui Qin Jason Eisner	4
+	WARP: Word-level Adversarial ReProgramming	2021	Karen Hambardzumyan Hrant Khachatrian Jonathan May	4
+ PDF Chat	ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora	2021	Xuan Ouyang Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu Haifeng Wang	4
+ PDF Chat	<scp>Canine</scp>: Pre-training an Efficient Tokenization-Free Encoder for Language Representation	2022	Jonathan H. Clark Dan Garrette Iulia Turc John Wieting	4
+	Adafactor: Adaptive Learning Rates with Sublinear Memory Cost	2018	Noam Shazeer Mitchell Stern	4
+	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	2018	Jacob Devlin Ming‐Wei Chang Kenton Lee Kristina Toutanova	4
+	AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts	2020	Taylor Shin Yasaman Razeghi Robert L. Logan Eric Wallace Sameer Singh	4
+	SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine	2017	Matthew Dunn Levent Sagun Mike Higgins V. Uğur Güney Volkan Cirik Kyunghyun Cho	4
+ PDF Chat	TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension	2017	Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer	4
+ PDF Chat	RACE: Large-scale ReAding Comprehension Dataset From Examinations	2017	Guokun Lai Qizhe Xie Hanxiao Liu Yiming Yang Eduard Hovy	4
+ PDF Chat	FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding	2021	Yuwei Fang Shuohang Wang Zhe Gan Siqi Sun Jun Liu	4
+	Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges	2019	Naveen Arivazhagan Ankur Bapna Orhan Fırat Dmitry Lepikhin Melvin Johnson Maxim Krikun Mia Xu Chen Yuan Cao George Foster Colin Cherry	4
+	GLU Variants Improve Transformer.	2020	Noam Shazeer	4
+ PDF Chat	Improving Neural Machine Translation Models with Monolingual Data	2016	Rico Sennrich Barry Haddow Alexandra Birch	4
+	ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension	2018	Sheng Zhang Xiaodong Liu Jun Liu Jianfeng Gao Kevin Duh Benjamin Van Durme	4
+	Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks	2020	Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith	4