home..
GUpdater + MEMIT
huyi / July 2023
1. Method
two-stage method:
2. Task and Dataset
Task
- read texts describing the updates,
- according to which modify the knowledge graph,
- and then insert the added triples into GPT.
Dataset
modify the family tree dataset, only include one kind of update: change the name of one entity
text: {entity_old} changes his or her name into {entity_new}.
3. Results
divide the task into two stages
- first stage: test the ability of GUpdater to extract the added and removed triples given the texts and the original KGs
- second stage: test the performance of MEMIT regarding inserting the added triples into GPT
- NOTICE: that MEMIT can only add triples into GPT and is unable to delete triples in GPT, so in the second stage, we do not test its ability to remove triples.
3.1 Triple Extraction: GUpdater
- Result: the accuracy of added triples is low
- Reason: the new triple (like Dylan in the showed figure) does not have any link to the other entities in the original KG, so it can be hard to form links between the new triple and the other triples
3.2 Knowledge Insertion: MEMIT
- Result:
- MEMIT is not powerful enough to insert memories into GPT.
- However, bigger models perform better.
4. Some Other Thoughts
Flaws of MEMIT
- A main weak point of MEMIT is that it cannot remove triples in GPT
- MEMIT cannot deal with one-to-many mapping. To be more specific, triples like (head, relation, ?) can maps to many tail entities, for example, when the relation is “teammate”. However, MEMIT cannot deal with such situations.
Future Work
The modeling of “key-value mapping” in MEMIT may be too hard. We may model such relations between head entities and tail entities in a softer way like works in the field of KG, such as Q2B and FuzzyQE.