Tag: Natural Language Processing

Ich bin ein berliner

Brain to Speech

This project focuses on decoding neural signals from brain areas involved in speech production (ventral premotor cortex and Broca’s area) into text by combining brain feature extractors and ASR architectures. Whereas previous research disregards the semantic regions (e.g. Area 44), this project explores combining semantic and motor (articulatory) signals in the brain. The dataset comes from a neuroprosthesis study, and the result will help people with paralysis to communicate.

Image credit: arXiv

Semi-supervised adversarial text generation based on seq2seq models

Hieu Le, Dieu-Thu Le, Verena Weber, Chris Church, Kay Rottmann, Melanie Bradford, Sang (Peter) Chin

Publication:

Empirical Methods in Natural Language Processing (EMNLP) (Conference Paper)

URL
Cite

@Inproceedings{Le2022,
author = {Hieu Le and Dieu-Thu Le and Verena Weber and Chris Church and Kay Rottmann and Melanie Bradford and Peter Chin},
title = {Semi-supervised adversarial text generation based on seq2seq models},
year = {2022},
url = {https://www.amazon.science/publications/semi-supervised-adversarial-text-generation-based-on-seq2seq-models},
booktitle = {EMNLP 2022},
}

Referring Expression Problem

The problem of referring expression is a more domain specific area of image captioning with the goal of describing a sub-region of a given image. Rational Speech Act (RSA) framework is a probabilistic reasoning approach that can generate sentences based on game theory systems of speaker – listener. The advantage of RSA is its explainability – namely answer the question of why a speaking agent choosing a specific word/phrase over another. Can RSA be applied to referring expression problem to generate a better/more explainable description?

Image credit: wmonroeiv.github.io