graviti logo产品公开数据集关于我们
登录
219
0
8
Children Book Test
创建来自Hello Dataset / Robert
概要
活动

Overview

Children’s Book Test (CBT), designed to measure directly how well language models can exploit wider linguistic context. The CBT is built from books that are freely available thanks to Project Gutenberg. Details and baseline results on this dataset can be found in the paper:

Felix Hill, Antoine Bordes, Sumit Chopra and Jason Weston. The Goldilocks Principle: Reading Children’s Books with Explicit Memory Representations, arXiv:1511.02301.

After allocating books to either training, validation or test sets, we formed example ‘questions’ from chapters in the book by enumerating 21 consecutive sentences. In each question, the first 20 sentences form the context, and a word is removed from the 21st sentence, which becomes the query. Models must identify the answer word among a selection of 10 candidate answers appearing in the context sentences and the query. For finer-grained analyses, we evaluated four classes of question by removing distinct types of word: Named Entities, (Common) Nouns, Verbs and Prepositions

Here is an example of question (context + query) from Alice in Wonderland by Lewis Carroll:

Context:
1 So they had to fall a long way .
2 So they got their tails fast in their mouths .
3 So they could n't get them out again .
4 That 's all .
5 `` Thank you , " said Alice , `` it 's very interesting .
6 I never knew so much about a whiting before . "
7 `` I can tell you more than that , if you like , " said the Gryphon .
8 `` Do you know why it 's called a whiting ? "
9 `` I never thought about it , " said Alice .
10 `` Why ? "
11 `` IT DOES THE BOOTS AND SHOES . '
12 the Gryphon replied very solemnly .
13 Alice was thoroughly puzzled .
14 `` Does the boots and shoes ! "
15 she repeated in a wondering tone .
16 `` Why , what are YOUR shoes done with ? "
17 said the Gryphon .
18 `` I mean , what makes them so shiny ? "
19 Alice looked down at them , and considered a little before she gave her answer .
20 `` They 're done with blacking , I believe . "

Query: `` Boots and shoes under the sea , " the XXXXX went on in a
deep voice , `` are done with a whiting ".

Candidates: Alice|BOOTS|Gryphon|SHOES|answer|fall|mouths|tone|way|whiting

Answer: gryphon

Citation

Please use the following citation when referencing the dataset:

@article{hill2015goldilocks,
  title={The goldilocks principle: Reading children's books with explicit memory representations},
  author={Hill, Felix and Bordes, Antoine and Chopra, Sumit and Weston, Jason},
  journal={arXiv preprint arXiv:1511.02301},
  year={2015}
}
数据集信息
应用场景NLP
标注类型Text
LicenseUnknown
更新时间2021-03-24 22:53:02
数据概要
数据格式Text
数据数量0
文件大小115KB
标注数量0
版权归属方
Facebook Research
标注方
未知
了解更多和支持
立即开始构建AI
免费开始联系我们