publications

2022

1. Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models

   Clara Na, Sanket Vaibhav Mehta, and Emma Strubell

   In Findings of the Association for Computational Linguistics: EMNLP 2022. Dec, 2022.

   HTML PDF
2. A Survey of Active Learning for Natural Language Processing

   Zhisong Zhang,  Emma Strubell, and Eduard Hovy

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Dec, 2022.

   HTML PDF
3. Transfer Learning from Semantic Role Labeling to Event Argument Extraction with Template-based Slot Querying

   Zhisong Zhang,  Emma Strubell, and Eduard Hovy

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Dec, 2022.
4. Bridging Fairness and Environmental Sustainability in Natural Language Processing

   Marius Hessenthaler,  Emma Strubell, Dirk Hovy, and Anne Lauscher

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Dec, 2022.
5. Evaluating Gender Bias Transfer from Film Data

   Amanda Bertsch, Ashley Oh, Sanika Natu, Swetha Gangu, Alan W. Black, and Emma Strubell

   In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP). Jul, 2022.

   Abs

   Films are a rich source of data for natural language processing. OpenSubtitles (Lison and Tiedemann, 2016) is a popular movie script dataset, used for training models for tasks such as machine translation and dialogue generation. However, movies often contain biases that reflect society at the time, and these biases may be introduced during pre-training and influence downstream models. We perform sentiment analysis on template infilling (Kurita et al., 2019) and the Sentence Embedding Association Test (May et al., 2019) to measure how BERT-based language models change after continued pre-training on OpenSubtitles. We consider gender bias as a primary motivating case for this analysis, while also measuring other social biases such as disability. We show that sentiment analysis on template infilling is not an effective measure of bias due to the rarity of disability and gender identifying tokens in the movie dialogue. We extend our analysis to a longitudinal study of bias in film dialogue over the last 110 years and find that continued pre-training on OpenSubtitles encodes additional bias into BERT. We show that BERT learns associations that reflect the biases and representation of each film era, suggesting that additional care must be taken when using historical data.

6. Measuring the Carbon Intensity of AI in Cloud Instances

   Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz,  Emma Strubell, Alexandra Sasha Luccioni, Noah S. Smith, Nicole Decario, and Will Buchanan

   In ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT). Jun, 2022.

   HTML PDF
7. Improving Compositional Generalization with Self-Training for Data-to-Text Generation

   Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay, Mihir Kale, Ankur Parikh, and Emma Strubell

   In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. May, 2022.

   HTML PDF
8. Aligning artificial intelligence with climate change mitigation

   Lynn H Kaack, Priya L Donti,  Emma Strubell, George Kamiya, Felix Creutzig, and David Rolnick

   Nature Climate Change. May, 2022.

   HTML PDF

2021

1. On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling

   Zhisong Zhang,  Emma Strubell, and Eduard Hovy

   In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Nov, 2021.

   HTML PDF Code
2. Comparing Span Extraction Methods for Semantic Role Labeling

   Zhisong Zhang,  Emma Strubell, and Eduard Hovy

   In Proceedings of the 5th Workshop on Structured Prediction for NLP (SPNLP 2021). Aug, 2021.

   HTML PDF Code
3. Spotlight talk

   An empirical investigation of the role of pre-training in lifelong learning

   Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell

   In ICML Theory and Foundation of Continual Learning Workshop. Aug, 2021.

   HTML PDF
4. WiFiMod: Transformer-Based Indoor Human Mobility Modeling Using Passive Sensing

   Amee Trivedi, Kate Silverstein,  Emma Strubell, Prashant Shenoy, and Mohit Iyyer

   In ACM SIGCAS Conference on Computing and Sustainable Societies (COMPASS). Aug, 2021.

   Abs HTML PDF

   Modeling human mobility has a wide range of applications from urban planning to simulations of disease spread. It is well known that humans spend 80% of their time indoors but modeling indoor human mobility is challenging due to three main reasons: (i) the absence of easily acquirable, reliable, low-cost indoor mobility datasets, (ii) high prediction space in modeling the frequent indoor mobility, and (iii) multi-scalar periodicity and correlations in mobility. To deal with all these challenges, we propose WiFiMod, a Transformer-based, data-driven approach that models indoor human mobility at multiple spatial scales using WiFi system logs. WiFiMod takes as input enterprise WiFi system logs to extract human mobility trajectories from smartphone digital traces. Next, for each extracted trajectory, we identify the mobility features at multiple spatial scales, macro and micro, to design a multi-modal embedding Transformer that predicts user mobility for several hours to an entire day across multiple spatial granularities. Multi-modal embedding captures the mobility periodicity and correlations across various scales while Transformers capture long term mobility dependencies boosting model prediction performance. This approach significantly reduces the prediction space by first predicting macro mobility, then modeling indoor scale mobility, micro mobility, conditioned on the estimated macro mobility distribution, thereby using the topological constraint of the macro-scale. Experimental results show that WiFiMod achieves a prediction accuracy of at least 10% points higher than the current state-of-art models. Additionally, we present 3 real-world applications of WiFiMod - (i) predict high density hot pockets and space utilization for policy making decisions for COVID19 or ILI, (ii) generate a realistic simulation of indoor mobility data to simulate spread of diseases, (iii) design personal assistants.

2020

1. Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks

   Edward Kim, Zach Jensen, Alexander Grootel, Kevin Huang, Matthew Staib, Sheshera Mysore, Haw-Shiuan Chang,  Emma Strubell, Andrew McCallum, Stefanie Jegelka, and Elsa Olivetti

   Journal of Chemical Information and Modeling. Aug, 2020.

   HTML
2. Artificial Intelligence and Climate Change: Opportunities, considerations, and policy levers to align AI with climate change goals

   Lynn H. Kaack, Priya Donti,  Emma Strubell, and David Rolnick

   Invited policy brief, Heinrich Böll Foundation. Dec, 2020.

   HTML PDF
3. Energy and Policy Considerations for Modern Deep Learning Research

   Emma Strubell, Ananya Ganesh, and Andrew McCallum

   Proceedings of the AAAI Conference on Artificial Intelligence. Apr, 2020.

   HTML PDF

2019

1. The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures

   Sheshera Mysore, Zachary Jensen, Edward Kim, Kevin Huang, Haw-Shiuan Chang,  Emma Strubell, Jeffrey Flanigan, Andrew McCallum, and Elsa Olivetti

   In Proceedings of the 13th Linguistic Annotation Workshop. Aug, 2019.

   HTML PDF
2. AI Impact Prize

   Energy and Policy Considerations for Deep Learning in NLP

   Emma Strubell, Ananya Ganesh, and Andrew McCallum

   In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Jul, 2019.

   HTML PDF
3. Emma Strubell. 2019. Machine Learning Models for Efficient and Robust Natural Language Processing. Ph.D. thesis, University of Massachusetts Amherst, September.

   HTML PDF

2018

1. Syntax Helps ELMo Understand Semantics: Is Syntax Still Relevant in a Deep Neural Architecture for SRL?

   Emma Strubell, and Andrew McCallum

   In Proceedings of the Workshop on the Relevance of Linguistic Structure in Neural Architectures for NLP. Jul, 2018.

   HTML PDF
2. Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction

   Patrick Verga,  Emma Strubell, and Andrew McCallum

   In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Jun, 2018.

   HTML PDF Code
3. Syntax Helps ELMo Understand Semantics: Is Syntax Still Relevant in a Deep Neural Architecture for SRL?

   Emma Strubell, and Andrew McCallum

   In Proceedings of the Workshop on the Relevance of Linguistic Structure in Neural Architectures for NLP. Jun, 2018.

   HTML PDF
4. Best Paper

   Linguistically-Informed Self-Attention for Semantic Role Labeling

   Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum

   In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Oct, 2018.

   HTML PDF Code
5. Multi-Task Learning For Parsing The Alexa Meaning Representation Language

   Vittorio Perera, Tagyoung Chung, Thomas Kollar, and Emma Strubell

   Proceedings of the AAAI Conference on Artificial Intelligence. Apr, 2018.

   HTML PDF

2017

1. Dependency Parsing with Dilated Iterated Graph CNNs

   Emma Strubell, and Andrew McCallum

   In Proceedings of the 2nd Workshop on Structured Prediction for Natural Language Processing. Sep, 2017.

   HTML PDF
2. Fast and Accurate Entity Recognition with Iterated Dilated Convolutions

   Emma Strubell, Patrick Verga, David Belanger, and Andrew McCallum

   In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Sep, 2017.

   HTML PDF Code
3. An epidemiological model of internet worms with hierarchical dispersal and spatial clustering of hosts

   David E. Hiebeler, Andrew Audibert,  Emma Strubell, and Isaac J. Michaud

   Journal of Theoretical Biology. Sep, 2017.

   HTML
4. Machine-learned and codified synthesis parameters of oxide materials

   Edward Kim, Kevin Huang, Alex Tomala, Sara Matthews,  Emma Strubell, Adam Saunders, Andrew McCallum, and Elsa Olivetti

   Nature Scientific Data. Sep, 2017.

   HTML PDF Code
5. Spotlight talk

   Automatically Extracting Action Graphs From Materials Science Synthesis Procedures

   Sheshera Mysore, Edward Kim,  Emma Strubell, Ao Liu, Haw-Shiuan Chang, Srikrishna Kompella, Kevin Huang, Andrew McCallum, and Elsa Olivetti

   In NIPS Workshop on Machine Learning for Molecules and Materials. Dec, 2017.

   HTML PDF
6. Attending to All Mention Pairs for Full Abstract Biological Relation Extraction

   Patrick Verga,  Emma Strubell, Ofer Shai, and Andrew McCallum

   In 6th Workshop on Automated Knowledge Base Construction (AKBC). Dec, 2017.

   HTML PDF

2016

1. Multilingual Relation Extraction using Compositional Universal Schema

   Patrick Verga, David Belanger,  Emma Strubell, Benjamin Roth, and Andrew McCallum

   In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Jun, 2016.

   HTML PDF Code
2. Extracting Multilingual Relations under Limited Resources: TAC 2016 Cold-Start KB construction and Slot-Filling using Compositional Universal Schema

   Haw-Shiuan Chang, Abdurrahman Munir, Ao Liu, Johnny Tian-Zheng Wei, Aaron Traylor, Ajay Nagesh, Nicholas Monath, Patrick Verga,  Emma Strubell, and Andrew McCallum

   In Text Analysis Conference (Knowledge Base Population Track) ’16 Workshop (TAC KBP). Nov, 2016.

   HTML PDF

2015

1. Outstanding Paper

   Learning Dynamic Feature Selection for Fast Sequential Prediction

   Emma Strubell, Luke Vilnis, Kate Silverstein, and Andrew McCallum

   In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Jul, 2015.

   HTML PDF
2. Building Knowledge Bases with Universal Schema: Cold Start and Slot-Filling Approaches

   Benjamin Roth, Nicholas Monath, David Belanger,  Emma Strubell, Patrick Verga, and Andrew McCallum

   In Text Analysis Conference (Knowledge Base Population Track) ’15 Workshop (TAC KBP). Nov, 2015.

   HTML PDF

2014

1. Training for Fast Sequential Prediction Using Dynamic Feature Selection

   Emma Strubell, Luke Vilnis, and Andrew McCallum

   In NIPS Workshop on Modern Machine Learning and NLP (NIPS WS). Dec, 2014.

   HTML PDF
2. Minimally Supervised Event Argument Extraction using Universal Schema

   Benjamin Roth,  Emma Strubell, Katherine Silverstein, and Andrew McCallum

   In 4th Workshop on Automated Knowledge Base Construction (AKBC). Dec, 2014.

   HTML PDF
3. Universal Schema for Slot-Filling, Cold-Start KBP and Event Argument Extraction: UMassIESL at TAC KBP 2014

   Benjamin Roth,  Emma Strubell, John Sullivan, Lakshmi Vikraman, Katherine Silverstein, and Andrew McCallum

   In Text Analysis Conference (Knowledge Base Population Track) ’14 Workshop (TAC KBP). Nov, 2014.

2012

1. Emma Strubell. 2012. Modeling the Spread of Biologically-Inspired Internet Worms. Ph.D. thesis, University of Maine, May. Undergraduate honors thesis.

   HTML PDF