Tensorflow 中 crf

Tensorflow 中 crf_decode 和 viterbi_decode 的使用

tf.contrib.crf势函数（potential）：训练过程（参数估计问题）解码过程（序列问题）

tf.contrib.crf

官方文档：https://www.tensorflow.org/api_docs/python/tf/contrib/crf?hl=en Classes

class CrfDecodeBackwardRnnCell: Computes backward decoding in a linear-chain CRF. class CrfDecodeForwardRnnCell: Computes the forward decoding in a linear-chain CRF. class CrfForwardRnnCell: Computes the alpha values in a linear-chain CRF.

Functions

crf_binary_score(...): Computes the binary scores of tag sequences. crf_decode(...): Decode the highest scoring sequence of tags in TensorFlow. crf_log_likelihood(...): Computes the log-likelihood of tag sequences in a CRF. crf_log_norm(...): Computes the normalization for a CRF. crf_multitag_sequence_score(...): Computes the unnormalized score of all tag sequences matching tag_bitmap. crf_sequence_score(...): Computes the unnormalized score for a tag sequence. crf_unary_score(...): Computes the unary scores of tag sequences. viterbi_decode(...): Decode the highest scoring sequence of tags outside of TensorFlow.

势函数（potential）：

unary potential：一元势函数：衡量单个状态（某个字，或某个像素）的似然标签 pairwise potential：二元势函数：衡量相邻状态之间的势能

http://ai.stanford.edu/~pawan/teaching/optimization/lecture5.pdf https://blog.csdn.net/lansatiankongxxc/article/details/45590545

训练过程（参数估计问题）

Tensorflow 中 tf.contrib.crf.crf_log_likelihood 用于计算crf_loss， 在 bi-lstm + crf 或 idcnn + crf 结构中作为crf的网络的损失函数。

对于损失函数的实现细节，可以参考这篇文章：CRF/Seq2Seq/CTC的Loss实现对比

tf.contrib.crf.crf_log_likelihood( inputs, tag_indices, sequence_lengths, transition_params=None ) Defined in tensorflow/contrib/crf/python/ops/crf.py. Computes the log-likelihood of tag sequences in a CRF.

参数:

inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials to use as input to the CRF layer.tag_indices: A [batch_size, max_seq_len] matrix of tag indices for which we compute the log-likelihood.sequence_lengths: A [batch_size] vector of true sequence lengths.transition_params: A [num_tags, num_tags] transition matrix, if available.

返回值:

log_likelihood: A [batch_size] Tensor containing the log-likelihood of each example, given the sequence of tag indices.transition_params: A [num_tags, num_tags] transition matrix. This is either provided by the caller or created in this function.

解码过程（序列问题）

看tensorflow的文档，说明 viterbi_decode 和 crf_decode 实现了相同功能，前者是numpy的实现，后者是 tensor 的实现，本文为了验证两者的解码结果是一致的。

测试环境：python3.6 + tensorflow1.8

import tensorflow as tf from tensorflow.contrib.crf import viterbi_decode from tensorflow.contrib.crf import crf_decode score = [[ [1, 2, 3], [2, 1, 3], [1, 3, 2], [3, 2, 1] ]] # (batch_size, time_step, num_tabs) transition = [ [2, 1, 3], [1, 3, 2], [3, 2, 1] ] # (num_tabs, num_tabs) lengths = [len(score[0])] # (batch_size, time_step) # numpy print("[numpy]") np_op = viterbi_decode( score=np.array(score[0]), transition_params=np.array(transition)) print(np_op[0]) print(np_op[1]) print("=============") # tensorflow score_t = tf.constant(score, dtype=tf.int64) transition_t = tf.constant(transition, dtype=tf.int64) lengths_t = tf.constant(lengths, dtype=tf.int64) tf_op = crf_decode( potentials=score_t, transition_params=transition_t, sequence_length=lengths_t) with tf.Session() as sess: paths_tf, scores_tf = sess.run(tf_op) print("[tensorflow]") print(paths_tf) print(scores_tf)

输出：

[numpy] [2, 0, 2, 0] 19 ============= [tensorflow] [[2 0 2 0]] [19]

可见结果是一致的，说明 crf_decode 就是 viterbi_decode 的tensorflow版本。

以下给出结果的计算说明：

tensorflow中两个函数的文档：https://www.tensorflow.org/api_docs/python/tf/contrib/crf

tf.contrib.crf.crf_decode( potentials, transition_params, sequence_length ) Defined in tensorflow/contrib/crf/python/ops/crf.py. Decode the highest scoring sequence of tags in TensorFlow. This is a function for tensor. Args: potentials: A [batch_size, max_seq_len, num_tags] tensor of unary potentials. transition_params: A [num_tags, num_tags] matrix of binary potentials. sequence_length: A [batch_size] vector of true sequence lengths. Returns: decode_tags: A [batch_size, max_seq_len] matrix, with dtype tf.int32. Contains the highest scoring tag indices. best_score: A [batch_size] vector, containing the score of decode_tags. tf.contrib.crf.viterbi_decode( score, transition_params ) Defined in tensorflow/contrib/crf/python/ops/crf.py. Decode the highest scoring sequence of tags outside of TensorFlow. This should only be used at test time. Args: score: A [seq_len, num_tags] matrix of unary potentials. transition_params: A [num_tags, num_tags] matrix of binary potentials. Returns: viterbi: A [seq_len] list of integers containing the highest scoring tag indices. viterbi_score: A float containing the score for the Viterbi sequence.