Plausibility
What is preferred is considered the most plausible, but what is
unusual might still be highly plausible. In our definition,
plausibility therefore (i) exceeds the boundaries of (selectional)
preference. Further, plausibility (ii) is a matter of degree as the
preferred is considered more plausible. In turn, what is unusual is
still considered plausible albeit to a lesser degree. Moreover,
plausibility (iii) captures non-surprisal in a given context, and (iv)
denotes what is generally likely, but not necessarily attested in a
given corpus.
Dataset for Plausibility of Vehicle Components
Evaluation dataset for the task of plausible material extraction in the vehicle repair domain. The dataset contains 100 sampled full components, and 100 sampled full+head components. The components were used for query template analysis and final evaluation.See here on how to obtain the data.
Reference:
Annerose Eichel, Helena Schlipf, Sabine Schulte im Walde (2023)
Made of Steel? Learning Plausible Materials for Components in the Vehicle Repair Domain
In: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Dubrovnik, Croatia.
PAP: Dataset for Physical and Abstract Plausibility
We present a novel dataset for physical and abstract plausibility of events in English. Based on naturally occurring sentences extracted from Wikipedia, we infiltrate degrees of abstractness, and automatically generate perturbed pseudo-implausible events. We annotate a filtered and balanced subset for plausibility using crowdsourcing, and perform extensive cleansing to ensure annotation quality. In-depth quantitative analyses indicate that annotators favor plausibility over implausibility and disagree more on implausible events. Furthermore, our plausibility dataset is the first to capture abstractness in events to the same extent as concreteness, and we find that event abstractness has an impact on plausibility ratings: more concrete event participants trigger a perception of implausibility.See here
on how to obtain the data.
Reference:
Annerose Eichel, Sabine Schulte im Walde (2023)
A Dataset for Physical and Abstract Plausibility and Sources of Human Disagreement
In: Proceedings of the 17th Linguistic Annotation Workshop (LAW). Toronto, Canada.