10-12 June, 2024
Artificial Neural Networks (ANNs) have proven to be powerful learning devices for language-related tasks, as demonstrated by recent progress in artificial intelligence driven by large, Transformer-based language models. But how can ANNs inform us about human language learning and processing? Our three-day
workshop brings together researchers working on cognitively motivated and linguistic questions in studying the language processing mechanisms and learning trajectories of ANNs.
For the first two days of the programme, we hope to stimulate discussion on the workshop theme through contributed presentations from our workshop participants and keynote speakers. The final day is focussed on active interactions and collaboration between participants, through small-scale tutorials and joint group work on a collaborative task. See our provisional programme below for more information and currently confirmed participants!
Registration
Registration is now open until May 24th! Please register here if you are interested in attending. Participation in the workshop is free of charge but the number of participants is limited. Priority will be given to junior researchers (PhD students and postdocs) and researchers who take part in the poster session and the collaborative task if we need to limit the number of attendees.
Organizers
Tamar Johnson (t.johnson@uva.nl)
Marianne de Heer Kloots (m.l.s.deheerkloots@uva.nl)
Venue
Institute for Logic, Language and Computation
SustainaLab event space, Amsterdam Science Park campus, University of Amsterdam
Confirmed keynote speakers:
Arianna Bisazza (University of Groningen)
Can modern Language Models be truly polyglot? Language learnability and inequalities in NLP
Eva Portelance (Mila; HEC Montréal)
What neural networks can teach us about how we learn language
Ethan Wilcox (ETH Zürich)
Using artificial neural networks to study human language processing: Two case studies and a warningNeural network language models are pure prediction engines, they have no communicative intent, and they do not learn language through social interactions. Despite this, I argue that they can be used to study human language processing, in particular, to empirically evaluate theories that are based on probability distributions over words. In the first half of this talk, I discuss two case studies in this vein, focusing on psycholinguistic theories of incremental processing, as well as regressions, or backward saccades between words. In the second half of the talk, I take a step back and discuss the impact of scaling for the usefulness of ANNs in psycholinguistics research. Scaling is the trend toward producing ever-larger models, both in terms of parameter counts and in terms of the amount of data they are trained on. While largely beneficial to performance on downstream benchmarking tasks, scaling has several downsides for computational psycholinguistics. I will discuss the scientific and practical challenges presented by scaling for neural network modeling, as well as the benefits that would result from human-scale language modeling research.
Programme
Note that time slots in the schedule below are still preliminary and subject to change. We hope to publish our finalized programme within the next few weeks, check back soon!
Monday June 10th: First day of lecture, talks and posters
| 13.30 – 14.00 | Walk-in / registration | ||
| 14.00 – 14.15 | Opening | ||
| 14.15 – 15.15 | Keynote lecture Arianna Bisazza – Can modern Language Models be truly polyglot? Language learnability and inequalities in NLP | ||
| 15.15 – 16.15 | Talks Confirmed speakers: Tessa Verhoef (Leiden University), Lukas Galke (MPI Nijmegen) | ||
| 16:15 – 16:30 | Break | ||
| 16:30 – 18:00 | Poster session | ||
| 19:00 – 21:00 | Workshop dinner |
Tuesday June 11th: Second day of lectures, talks and discussions
| 09.30 – 10.30 | Keynote lecture Eva Portelance – What neural networks can teach us about how we learn language |
| 10.30 – 12.45 | Talk session and moderated discussion on language learning Confirmed speakers: Raquel Alhama (University of Amsterdam), Kyle Mahowald (University of Texas at Austin), Yevgen Matusevych (University of Groningen) |
| 12.45 – 14.00 | Lunch |
| 14.00 – 15.00 | Keynote lecture Ethan Wilcox – Using artificial neural networks to study human language processing: Two case studies and a warning |
| 15.00 – 16.00 | Talk session on modelling language processing Confirmed speakers: Irene Winther (University of Edinburgh) Pierre Orhan (École Normale Supérieure) |
| 16.00 – 16.15 | Break |
| 16.15 – 16.45 | Position statements by Micha Heilbron (University of Amsterdam) and Stefan Frank (Radboud University Nijmegen) |
| 16.45 – 17.15 | Discussion on using ANNs in modelling language processing Moderator: Raquel Fernández (University of Amsterdam) |
| 17.15 – 17.30 | Closing |
Wednesday June 12th: Interactions on learning trajectories in smaller- and larger-scale models
| 09.30 – 09.45 | Introduction to the tutorials and collaborative tasks |
| 09.45 – 10.30 | First tutorial by Henry Conklin (University of Edinburgh) Language learning as regularization: A non-parametric probing method for studying the emergence of structured representations over model training |
| 10.30 – 11.15 | Second tutorial by Oskar van der Wal & Marianne de Heer Kloots (University of Amsterdam) What’s in a developmental phase? Training dynamics & Behavioural characterizations of grammar learning |
| 11.15 – 12.00 | Split up in groups & brainstorm |
| 12.00 – 13.00 | Lunch |
| 13.00 – 17.30 | Group work on collaborative tasks & Discussion of findings |
| 17.30 | Drinks |
Acknowledgements
This workshop is supported by and organized as part of the Language in Interaction consortium (NWO Gravitation Grant 024.001.006). We are also very thankful to the SustainaLab for lending us their space, and to Jelle Zuidema and the ILLC office for organizational advice and support!
