Omni-word Feature and Soft Constraint
for Chinese Relation Extraction

In the ACE corpus, an entity is an object or set of objects in the world. An entity mention is a reference to an entity. The entity mention is annotated with its full extent and its head, referred to as the extend mention and the head mention respectively. The extent mention includes both the head and its modifiers. Each relation has two entities as arguments: Arg-1 and Arg-2, referred to as E1 and E2. A relation mention (or instance) is the embodiment of a relation. It is referred by the sentence (or clause) in which the relation is located in. In our work, we focus on the detection and recognition of relation mention.

Relation identification is handled as a classification problem. Entity-related information (e.g. head noun, entity type, subtype, CLASS, LDCTYPE, etc.) are supposed to be known and provided by the corpus. In our experiment, the entity type, subtype and the head noun are used.

All the employed features are simply classified into five categories: Entity Type and Subtype, Head Noun, Position Feature, POS Tag and Omni-word Feature. The first four are widely used. The last one is proposed in this paper and is discussed in detail.

Entity Type and Subtype: In ACE 2005 RDC Chinese corpus, there are 7 entity types (Person, Organization, GPE, Location, Facility, Weapon and Vehicle) and 44 subtypes (e.g. Group, Government, Continent, etc.).

Head Noun: The head noun (or head mention) of entity mention is manually annotated. This feature is useful and widely used.

Position Feature: The position structure between two entity mentions (extend mentions). Because the entity mentions can be nested, two entity mentions may have four coarse structures: “E1 is before E2”, “E1 is after E2”, “E1 nests in E2” and “E2 nests in E1”, encoded as: ‘E1_B_E2’, ‘E1_A_E2’, ‘E1_N_E2’ and ‘E2_N_E1’.

POS Tag: In our model, we use only the adjacent entity POS tags, which lie in two sides of the entity mention. These POS tags are labelled by the ICTCLAS package²²http://ictclas.org/. The POS tags are not used independently. It is encoded by combining the POS tag with the adjacent entity mention information. For example ‘E1_Right_n’ means that the right side of the first entity is a noun (“n”).

Omni-word Feature: The notion of “word” in Chinese is vague and has never played a role in the Chinese philological tradition []. Some Chinese segmentation performance has been reported precision scores above 95% []. However, for the same sentence, even native peoples in China often disagree on word boundaries []. Sproat et al. [] has showed that there is a consistence of 75% on the segmentation among different native Chinese speakers. The word-formation of Chinese also implies that the meanings of a compound word are made up, usually, by the meanings of words that contained in it []. So, fragments of phrase are also informative.

Because high precision can be received by using simple lexical features []. Making better use of such information is beneficial. In consideration of the Chinese characteristics, we use every potential word in a relation mention as the lexical features. {CJK}UTF8gbsn For example, relation mention ‘台北大安森林公园’ (Taipei Daan Forest Park) has a ”PART-WHOLE” relation type. The traditional segmentation method may generate four lexical features {‘台北’, ‘大安’, ‘森林’, ‘公园’}, which is a partition of the relation mention. On the other hand, the Omni-word feature denoting all the possible words in the relation mention may generate features as:

1. {‘台’, ‘北’, ‘大’, ‘安’, ‘森’, ‘林’, ‘公’, ‘园’, ‘台北’, ‘大安’, ‘森林’, ‘公园’, ‘森林公园’, ‘大安森林公园’}³³The generated Omni-word features dependent on the employed lexicon.

Most of these features are nested or overlapped mutually. So, the traditional character-based or word-based feature is only a subset of the Omni-word feature. To extract the Omni-word feature, only a lexicon is required, then scan the sentence to collect every word.

Because the number of lexicon entry determines the dimension of the feature space, performance of Omni-word feature is influenced by the lexicon being employed. In this paper, we generate the lexicon by merging two lexicons. The first lexicon is obtained by segmenting every relation instance using the ICTCLAS package, collecting very word produced by ICTCLAS. Because the ICTCLAS package was trained on annotated corpus containing many meaningful lexicon entries. We expect this lexicon to improve the performance. The second lexicon is the Lexicon Common Words in Contemporary Chinese⁴⁴Published by Ministry of Education of the People’s Republic of China in 2008, containing 56,008 entries..

Despite the Omni-word can be seen as a subset of n-Gram feature. It is not the same as the n-Gram feature. N-Gram features are more fragmented. In most of the instances, the n-Gram features have no semantic meanings attached to them, thus have varied distributions. Furthermore, for a single Chinese word, occurrences of 4 characters are frequent. Even 7 or more characters are not rare. Because Chinese has plenty of characters⁵⁵Currently, at least 13000 characters are used by native Chinese people. Modern Chinese Dictionary: http://www.cp.com.cn/, when the corpus becoming larger, the n-Gram (n¿4) method is difficult to be adopted. On the other hand, the Omni-word can avoid these problems and take advantages of Chinese characteristics (the word-formation and the ambiguity of word segmentation).

The structure information (or dependent information) of relation instance is critical for recognition. However, even in English, “deeper” analysis (e.g. logical syntactic relations or predicate-argument structure) may suffer from a worse performance caused by inaccurate chunking or parsing. Hence, the local dependency contexts around the relation arguments are more helpful []. Zhang et al. [] also showed that Path-enclosed Tree (PT) achieves the best performance in the kernel based relation extraction. In this field, the tree kernel based method commonly uses the parse tree to capture the structure information []. On the other hand, the feature based method usually uses the combined feature to capture such structure information [].

In the open relation extraction domain, syntactic and semantic constraints are widely employed to prevent the “semantic drift” problem. Such constraints can also be seen as structural constraint. Most of these constraints are hard constraints. Any relation instance violating these constraints (or below a predefined threshold) will be abandoned. For example, Agichtein and Gravano [] generates patterns according to a confidence threshold (${\tau }_t$). Fader et al. [] utilizes a confidence function. And Carlson et al. [] filters candidate instances and patterns using the number of times they co-occurs.

Deleting of relation instances is acceptable for open relation extraction because it always deals with a big data set. But it’s not suitable for traditional relation extraction, and will result in a low recall. Utilizing the notion of combined feature [], we replace the hard constraint by the soft constraint. Each soft constraint (combined feature) has a parameter trained by the classifier indicating the discrimination ability it has. No subjective or priori judgement is adopted to delete any potential determinative constraint (except for the reason of dimensionality reduction).

Most of the researches make use of the combined feature, but rarely analyze the influence of the approaches we combine them. In this paper, we use the soft constraint to model the local dependency. It is a subset of the combined feature, generated by four constraint conditions: singleton, position sensitive, bin sensitive and semantic pair . For every employed candidate feature, an appropriate constraint condition is selected to combine them with additional information to maximize the classification determination.

UTF8gbsn Singleton: A feature is employed as a singleton feature when it is used without combining with any information. In our experiments, only the position feature is used as singleton feature.

Position Sensitive: A position sensitive feature has a label indicating which entity mention it depends on. In our experiment, the Head noun and POS Tag are utilized as position sensitive features, which has been introduced in Section 3.1. For example, ‘台北_E1’ means that the head noun ‘台北’ depend on the first entity mention.

Semantic Pair: Semantic pair is generated by combining two semantic units. Two kinds of semantic pair are employed. Those are generated by combining two entity types or two entity subtypes into a semantic pair. For example, ‘Person_Location’ denotes that the type of the first relation argument is a “Person” (entity type) and the second is a “Location” (entity type). Semantic pair can capture both the semantic and structure information in a relation mention.

Bin Sensitive: In our study, Omni-word feature is not added as “bag of words”. To use the Omni-word feature, we segment each relation mention by two entity mentions. Together with the two entity mentions, we get five parts: “FIRST”, “MIDDLE”, “END”, “E1” and “E2” (or less, if the two entity mentions are nested). Each part is taken as an independent bin. A flag is used to distinguish them. For example, ‘台北_Bin_F’, ‘台北_Bin_E1’ and ‘台北_Bin_E’ mean that the lexicon entry ‘台北’ appears in three bins: the FIRST bin, the first entity mention (E1) bin and the END bin. They will be used as three independent features.

To sum up, among the five candidate feature sets, the position feature is used as a singleton feature. Both head noun and POS tag are position sensitive. Entity types and subtypes are employed as semantic pair. Only Omni-word feature is bin sensitive. In the following experiments, focusing on Chinese relation extraction, we will analyze the performance of candidate feature sets and study the influence of the constraint conditions.

sectionExperiments

In this section, methodologies of the Omni-word feature and the soft constraint are tested. Then they are compared with the state-of-the-art methods.

We use the ACE 2005 RDC Chinese corpus, which was collected from newswires, broadcasts and weblogs, containing 633 documents with 6 major relation types and 18 subtypes. There are 8,023 relations and 9,317 relation mentions. After deleting 5 documents containing wrong annotations⁶⁶DAVYZW_{20041230.1024, 20050110.1403, 20050111.1514, 20050127.1720, 20050201.1538}., we keep 9,244 relation mentions as positive instances.

UTF8gbsn To get the negative instances, each document is segmented into sentences⁷⁷The five punctuations are used as sentence boundaries: Period (。), Question mark (？), Exclamatory mark (！), Semicolon (；) and Comma (，).. Those sentences that do not contain any entity mention pair are deleted. For each of the remained sentences, we iteratively extract every entity mention pair as the arguments of relation instances for predicting. For example, suppose a sentence has three entity mentions: A,B and C. Because the relation arguments are order sensitive, six entity mention pairs can be generated: [A,B], [A,C], [B,C], [B,A], [C,A] and [C,B]. After discarding the entity mention pairs that were used as positive instances, we generated 93,283 negative relation instances labelled as “OTHER”. Then, we have 7 relation types and 19 subtypes.

A maximum entropy multi-class classifier is trained and tested on the generated relation instances. We adopt the five-fold cross validation for training and testing. Because we are interested in the 6 annotated major relation types and the 18 subtypes, we average the results of five runs on the 6 positive relation types (and 18 subtypes) as the final performance. F-score is computed by

To implement the maximum entropy model, the toolkit provided by Le [] is employed. The iteration is set to $30$.

Five candidate feature sets are employed to generate the combined features. The entity type and subtype, head noun, position feature are referred to as ${ℱ}_{thp}$⁸⁸“thp” is an acronym of “type, head, position”. Features in ${ℱ}_{thp}$ are the candidate features combined with the corresponding constraint conditions. The following ${ℱ}_{pos}$ and ${ℱ}_{ow}$ are the same.. The POS tags are referred to as ${ℱ}_{pos}$. The Omni-word feature set is denoted by ${ℱ}_{ow}$.

Table 1 gives the performance of our system on the 6 types and 18 subtypes. Note that, in this paper, bare numbers and numbers in the parentheses represent the results of the 6 types and the 18 subtypes respectively.

In Row 1, because ${ℱ}_{thp}$ are features directly obtained from annotated corpus, we take this performance as our referential performance. In Row 2, with only the ${ℱ}_{ow}$ feature, the F-score already reaches 77.74% in 6 types and 60.31% in 18 subtypes. The last row shows that adding the ${ℱ}_{pos}$ almost has no effect on the performance when both the ${ℱ}_{thp}$ and ${ℱ}_{ow}$ are in use. The results show that ${ℱ}_{ow}$ is effective for Chinese relation extraction.

The superiorities of Owni-word feature depend on three reasons. First, the specificity of Chinese word-formation indicates that the subphrases of Chinese word (or phrase) are also informative. Second, most of relation instances have limited context. The Owni-word feature, utilizing every possible word in them, is a better way to capture more information. Third, the entity mentions are manually annotated. They can precisely segment the relation instance into corresponding bins. Segmentation of bins bears the sentence structure information. Therefore, the Owni-word feature with bin information can make a better use of both the syntactic information and the local dependency.

Various systems were proposed for Chinese relation extraction. We mainly focus on systems trained and tested on the ACE corpus. Table 2 lists three systems.

Che et al. [] was implemented on the ACE 2004 corpus, with 2/3 data for training and 1/3 for testing. The performance was reported on 7 relation types: 6 major relation types and the none relation (or negative instance). Zhang et al. [] was based on the ACE 2005 corpus with 75% data for training and 25% for testing. Performances about the 7 types and 19 subtypes were given. Both of them are feature based methods. Liu et al. [] is a kernel based method evaluated on the ACE 2005 corpus. The five-fold cross validation was used and declared the performances on 6 relation types and 18 subtypes.

The data preprocessing makes differences from our experiments to others. In order to give a better comparison with the state-of-the-art methods, based on our experiment settings and data, we implement the two feature based methods proposed by Che et al. [] and Zhang et al. [] in Table 2. The results are shown in Table 3.

In Table 3, Ei ($i\in 1,2$) represents entity mention. “Order” in Che et al. [] denotes the position structure of entity mention pair. Four types of order are employed (the same as ours). $Wor{d}_{\begin{array}{c}+\hfill \\ -\hfill \end{array}Ei,k}$ and $PO{S}_{\begin{array}{c}+\hfill \\ -\hfill \end{array}Ei,k}$ are the words and POS of Ei, “$\begin{array}{c}+\hfill \\ -\hfill \end{array}k$” means that it is the $k$th word (of POS) after (+) or before (-) the corresponding entity mention. In this paper, $k=1$ and $k=2$ were set.

In Row 2, the “Uni-Gram” represents the Uni-gram features of internal and external character sequences. Internal character sequences are the four entity extend and head mentions. Five kinds of external character sequences are used: one In-Between character sequence between E1 and E2 and four character sequences around E1 and E2 in a given window size w_s. The w_s is set to 4. The “Bi-Gram” is the 2-gram feature of internal and external character sequences. Instead of the 4 position structures, the 9 position structures are used. Please refer to Zhang et al. [] for the details of these 9 position structures.

In Table 3, it is shown that our system outperforms other systems, in F-score, by 10% on 6 relation types and by 15% on 18 subtypes.

For researchers who are interested in our work, the source code of our system and our implementations of Che et al. [] and Zhang et al. [] are available at https://github.com/YPench/CRDC.