Python 自然语言处理 第一章

from nltk.book import * *** Introductory Examples for the NLTK Book *** Loading text1, ..., text9 and sent1, ..., sent9 Type the name of the text or sentence to view it. Type: 'texts()' or 'sents()' to list the materials. text1: Moby Dick by Herman Melville 1851 text2: Sense and Sensibility by Jane Austen 1811 text3: The Book of Genesis text4: Inaugural Address Corpus text5: Chat Corpus text6: Monty Python and the Holy Grail text7: Wall Street Journal text8: Personals Corpus text9: The Man Who Was Thursday by G . K . Chesterton 1908

注意：若报错，将nltk存在报错信息中的指定目录里

concordance()用来查找单词，只能查单个词

similar()用来查找与参数上下文相同的词都有哪些，参数为单个词。上下文相同的词在NLP中被称为相似词

common_contexts()用来查找多个词所具有的共同上下文，参数为list形式

generate()可以用文本中的词汇随机产生一段文本

text1.concordance("Dick") Displaying 25 of 84 matches: Dick by Herman Melville 1851 ] ETYMOLOGY must be the same that some call Moby Dick ." " Moby Dick ?" shouted Ahab . " D e that some call Moby Dick ." " Moby Dick ?" shouted Ahab . " Do ye know the w Death and devils ! men , it is Moby Dick ye have seen -- Moby Dick -- Moby Di it is Moby Dick ye have seen -- Moby Dick -- Moby Dick !" " Captain Ahab ," sa ck ye have seen -- Moby Dick -- Moby Dick !" " Captain Ahab ," said Starbuck , Captain Ahab , I have heard of Moby Dick -- but it was not Moby Dick that too of Moby Dick -- but it was not Moby Dick that took off thy leg ?" " Who told my hearties all round ; it was Moby Dick that dismasted me ; Moby Dick that b s Moby Dick that dismasted me ; Moby Dick that brought me to this dead stump I white whale ; a sharp lance for Moby Dick !" " God bless ye ," he seemed to ha white whale ? art not game for Moby Dick ?" " I am game for his crooked jaw , l whaleboat ' s bow -- Death to Moby Dick ! God hunt us all , if we do not hun hunt us all , if we do not hunt Moby Dick to his death !" The long , barbed st owels to feel fear ! CHAPTER 41 Moby Dick . I , Ishmael , was one of that crew ividualizing tidings concerning Moby Dick . It was hardly to be doubted , that on must have been no other than Moby Dick . Yet as of late the Sperm Whale fis ident ignorantly gave battle to Moby Dick ; such hunters , perhaps , for the m g and piling their terrors upon Moby Dick ; those things had gone far to shake ies , which eventually invested Moby Dick with new terrors unborrowed from any rmen recalled , in reference to Moby Dick , the earlier days of the Sperm Whal ngs were ready to give chase to Moby Dick ; and a still greater number who , c was the unearthly conceit that Moby Dick was ubiquitous ; that he had actuall their superstitions ; declaring Moby Dick not only ubiquitous , but immortal ( shaped lower jaw beneath him , Moby Dick had reaped away Ahab ' s leg , as a

len(text)获取text文本中长度（其中包括标点符号）（相当于语料）

set(text)获取text中不重复元素的个数（相当于字典）

sort(set(text))对set（text)中的内容进行排序（标点符号在前面，按字母顺序排序）

text.count(“word”)计算text中word出现的次数

链表:1.多个链表的链接 +

2.在链表中增加元素 append（）

3.索引、切片

sent1 ['Call', 'me', 'Ishmael', '.']

字符串与链表之间的转换：jion()和split

sentence=' '.join(['OH','MY','GOD']) print sentence OH MY GOD split_sentence=sentence.split() print split_sentence ['OH', 'MY', 'GOD'] saying=['After','all','is','said','and','done','more','is','said','than','done'] tokens=sorted(set(saying))[-2:] print tokens ['said', 'than'] fdist1=FreqDist(text1) print fdist1 <FreqDist with 19317 samples and 260819 outcomes> for key in fdist1: print key,fdist1[key] funereal 1 unscientific 1 divinely 2 foul 11 four 74 gag 2 prefix 1 ……