自然语言处理 Paddle NLP - 开放域对话系统-理论
常见的对话系统
- 个人助手
•Apple Siri, Amazon Alexa, Baidu Xiaodu - 客户服务•餐厅预定、商品咨询、债务催收
- 休闲娱乐
•Microsoft XiaoIce,Replika
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/xtgyikrr_2jzw.png)
放域对话系统
- 检索式系统
- 生成式系统
检索式系统
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/kjhmbhhv_9k64.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/bzvorwvj_flna.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/tlyexjln_v18l.png)
生成模型
- Sequence-to-sequence
- Generative Pre-training Models
Sequence-to-sequence
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/ztnlodhc_fazj.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/sswauxka_kw7y.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/zgwiocfy_3n7l.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/tpmykgwa_zpu5.png)
Generative Pre-training Models
- BERT 使用了双向注意力机制,每个token可以向左向右看到各个token
- GPT 使用了单向注意力机制(从左到右),是一个生成模型,每个 token只能看到自己和前他面的词,只能看到以前的,所以比较适合用来做生成任务,像平常说话也是一个词一个词的,Q:你有没有养宠物,A:我有一只猫,
- PrefixLM (Unified Transformer)使用了编码端双向、生成端单向的注意力机制,适合开放域的对话任务
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/xupsjnvh_dio4.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/asobvbck_ng25.png)
解码策略
-
Greedy Decoding
-
Sampling-based Decoding
-
每一步,选取最可能的单词(i.e. argmax)
-
把选取的单词补充到input中,继续下一步解码
-
停止条件: 继续解码直到产生[EOS] 或者达到了事先定义的最大生成长度
Sampling-based Decoding
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/oswujdkq_r29d.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/pouetyxy_z1uj.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/mruvwree_yipn.png)
解码策略–总结
- Greedy decoding 方法很简单,但效果一般较差
- Sampling-baseddecoding可以得到更多样化/随机的回复
•比较适合于开放式/创造性的生成任务,像对话、诗词、故事生成
•Top-k/p sampling 可以通过调节k/p,对生成的多样性进行控制
对话评估
- 自动评估: BLEU、Distinct
- 人工评估
BLEU
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/giszkmmz_bovq.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/xdhqevnh_5k5i.png)
Distinct
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/edqakwol_fhju.png)
人工评估
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/kygasuug_vino.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/pcevhhfa_9yn5.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/avpviavm_h2hy.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/mahwwrbl_p9dc.png)
开放域对话模型演进历史
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/sorkqoay_8fm0.png)
PLATO-2
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/owbgiebt_n9kx.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/qsehnnoi_d1o3.png)
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/ysiysyyu_36zk.png)
总结及展望
![](https://img.shuduke.com/static_img/cnblogs/blog/80824/202306/gwhnwufr_8pxz.png)
视频:https://aistudio.baidu.com/aistudio/course/introduce/24177?sharedLesson=1493709&sharedType=2&sharedUserId=2631487&ts=1687157269113
课件:https://aistudio.baidu.com/aistudio/course/introduce/24177?sharedLesson=1567921&sharedType=2&sharedUserId=2631487&ts=1687157254946