{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d2eb19ab",
   "metadata": {},
   "source": [
    "# 动手学大模型：大模型知识编辑\n",
    "导读: 语言模型的编辑方法和工具\n",
    "> 想操控语言模型在对指定知识的记忆？让我们选择合适的编辑方法，对特定知识进行编辑，并将对编辑后的模型进行验证！\n",
    "\n",
    "## 1. 本教程目标：\n",
    "\n",
    "- 熟悉使用EasyEdit工具包\n",
    "- 掌握语言模型的编辑方法（最简）\n",
    "- 了解不同类型的编辑方法的选型和应用场景"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa609334",
   "metadata": {},
   "source": [
    "## 2. 工作准备：\n",
    "### 2.1 了解EasyEdit\n",
    "\n",
    "https://github.com/zjunlp/EasyEdit\n",
    "\n",
    "EasyEdit 是一个用于编辑语言模型的 Python 包，如 GPT-J、Llama、GPT-NEO、GPT2、T5等，其目标是针对一个特定的知识有效地改变语言模型的行为，而不会对其他输入的性能产生负面影响，同时易于使用且易于扩展。\n",
    "\n",
    "EasyEdit 集成了现有的流行的编辑方法：\n",
    "![](./assets/1.png)\n",
    "\n",
    "### 2.2 主要框架\n",
    "\n",
    "![](./assets/2.png)\n",
    "EasyEdit包含一个统一的Editor、Method和Evaluate框架，分别代表编辑场景、编辑技术和评估方法。\n",
    "- Editor：描述工作场景，包含待编辑的模型，待编辑的知识，以及其他必要的超参数。\n",
    "- Method：所使用的具体知识编辑方法（例如ROME、MEND等）。\n",
    "- Evaluate：评估知识编辑性能的指标，包含了可靠性、通用性、局部性、可移植性。\n",
    "- Trainer：有些编辑方法需要一定的训练过程，由Trainer模块实现。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eabdb8e3",
   "metadata": {},
   "source": [
    "## 3. 安装环境：\n",
    "```\n",
    "git clone https://github.com/zjunlp/EasyEdit.git\n",
    "（可选）conda create -n EasyEdit python=3.9.7\n",
    "cd EasyEdit\n",
    "pip install -r requirements.txt\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a1e5b5",
   "metadata": {},
   "source": [
    "## 4. 编辑案例\n",
    "> 目标：改变GPT-2-XL的知识记忆，将梅西(Lionel Messi)的职业从足球改成篮球(football->basketball). \n",
    "步骤：\n",
    "- 选择编辑方法，准备参数\n",
    "- 准备知识编辑的数据\n",
    "- 实例化Editor\n",
    "- Run！\n",
    "下面以ROME方法为例具体介绍：\n",
    "### 4.1 ROME\n",
    "Jupiter Notebook: [https://colab.research.google.com/drive/1KkyWqyV3BjXCWfdrrgbR-QS3AAokVZbr?usp=sharing#scrollTo=zWfGkNb9FBJQ] \n",
    "- 选择编辑方法，准备参数\n",
    "  - 编辑方法选择为ROME，准备ROME和GPT2-XL所需要的参数。\n",
    "  - 例如：alg_name: \"ROME\"，model_name: \"./hugging_cache/gpt2-xl\"或为本地该模型的路径，\"device\": 使用的GPU序号\n",
    "  - 其余参数可保持默认\n",
    "![](./assets/3.png)\n",
    "- 准备知识编辑的数据\n",
    "    ```\n",
    "    prompts = ['Question:What sport does Lionel Messi play? Answer:'] # x_e\n",
    "    ground_truth = ['football'] # y\n",
    "    target_new = ['basketball'] # y_e\n",
    "    subject = ['Lionel Messi'] \n",
    "    ```\n",
    "- 实例化Editor，将准备好的参数传入BaseEditor类进行实例化，得到定制的Editor实例。\n",
    "    ```\n",
    "    hparams = ROMEHyperParams.from_hparams('./hparams/ROME/gpt2-xl.yaml')\n",
    "    editor=BaseEditor.from_hparams(hparams)\n",
    "    ```\n",
    "- Run! 调用editor的edit方法：\n",
    "    ```\n",
    "    metrics, edited_model, _ = editor.edit(\n",
    "        prompts=prompts,\n",
    "        ground_truth=ground_truth,\n",
    "        target_new=target_new,\n",
    "        subject=subject,\n",
    "        keep_original_weight=False\n",
    "    )\n",
    "    ```\n",
    "![](./assets/4.png)\n",
    "> 备注：首次编辑某个模型时会下载Wiki语料，并为该模型计算各层的状态（stats_dir: \"./data/stats\"）并存下，在后续的每次编辑中复用。因此，首次编辑有所耗时，确保网络通畅的情况下可耐心等待。\n",
    "### 4.2 验证与评估\n",
    "editor.edit会返回metrics（由EasyEdit的Evaluate模块计算）。形式为：\n",
    "![](./assets/5.png)\n",
    "要得到通用性、局部性、可移植性的数值，需要在edit方法中传入用于评估的数据。\n",
    "\n",
    "以局部性为例，会导致edit方法计算局部性的指标，即在locality_inputs上模型回答的正确率。\n",
    "```\n",
    "locality_inputs = {\n",
    "    'neighborhood':{\n",
    "        'prompt': ['Joseph Fischhof, the', 'Larry Bird is a professional', 'In Forssa, they understand'],\n",
    "        'ground_truth': ['piano', 'basketball', 'Finnish']\n",
    "    }\n",
    "}\n",
    "metrics, edited_model, _ = editor.edit(\n",
    "    prompts=prompts,\n",
    "    ground_truth=ground_truth,\n",
    "    target_new=target_new,\n",
    "    locality_inputs=locality_inputs,\n",
    "    keep_original_weight=False\n",
    ")\n",
    "```\n",
    "或者直接比较前后模型的generte行为。\n",
    "```\n",
    "generation_prompts = [\n",
    "    \"Lionel Messi, the\",\n",
    "    \"The law in Ikaalinen declares the language\"\n",
    "]\n",
    "\n",
    "model = GPT2LMHeadModel.from_pretrained('./hugging_cache/gpt2').to('cuda')\n",
    "batch = tokenizer(generation_prompts, return_tensors='pt', padding=True, max_length=30)\n",
    "\n",
    "pre_edit_outputs = model.generate(\n",
    "    input_ids=batch['input_ids'].to('cuda'),\n",
    "    attention_mask=batch['attention_mask'].to('cuda'),\n",
    "    max_new_tokens=3\n",
    ")\n",
    "post_edit_outputs = edited_model.generate(\n",
    "    input_ids=batch['input_ids'].to('cuda'),\n",
    "    attention_mask=batch['attention_mask'].to('cuda'),\n",
    "    max_new_tokens=3\n",
    ")\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fce071f0",
   "metadata": {},
   "source": [
    "## 5. 规模化的编辑（可选）\n",
    "### 5.1 Batch edit\n",
    "多条数据可以形成并列的列表同时传入edit方法进行batch edit，此时MEMIT为最佳方法。（https://colab.research.google.com/drive/1P1lVklP8bTyh8uxxSuHnHwB91i-1LW6Z）\n",
    "```\n",
    "prompts = ['Question:What sport does Lionel Messi play? Answer:',\n",
    "            'The law in Ikaalinen declares the language']\n",
    "ground_truth = ['football', 'Finnish']\n",
    "target_new = ['basketball', 'Swedish']\n",
    "subject = ['Lionel Messi', 'Ikaalinen']\n",
    "```\n",
    "### 5.2 Benchmark上测试\n",
    "- Counterfact\n",
    "- ZsRE\n",
    "```\n",
    "{\n",
    "    \"case_id\": 4402,\n",
    "    \"pararel_idx\": 11185,\n",
    "    \"requested_rewrite\": {\n",
    "      \"prompt\": \"{} debuted on\",\n",
    "      \"relation_id\": \"P449\",\n",
    "      \"target_new\": {\n",
    "        \"str\": \"CBS\",\n",
    "        \"id\": \"Q43380\"\n",
    "      },\n",
    "      \"target_true\": {\n",
    "        \"str\": \"MTV\",\n",
    "        \"id\": \"Q43359\"\n",
    "      },\n",
    "      \"subject\": \"Singled Out\"\n",
    "    },\n",
    "    \"paraphrase_prompts\": [\n",
    "      \"No one on the ground was injured.  v\",\n",
    "      \"The sex ratio was 1063. Singled Out is to debut on\"\n",
    "    ],\n",
    "    \"neighborhood_prompts\": [\n",
    "      \"Daria premieres on\",\n",
    "      \"Teen Wolf was originally aired on\",\n",
    "      \"Spider-Man: The New Animated Series was originally aired on\",\n",
    "      \"Celebrity Deathmatch premiered on\",\n",
    "      \"\\u00c6on Flux premiered on\",\n",
    "      \"My Super Psycho Sweet 16 premieres on\",\n",
    "      \"Daria was released on\",\n",
    "      \"Jersey Shore premiered on\",\n",
    "      \"Skins was originally aired on\",\n",
    "      \"All You've Got premiered on\"\n",
    "    ]\n",
    "  }\n",
    "  ```\n",
    "https://github.com/zjunlp/EasyEdit/blob/main/examples/run_zsre_llama2.py "
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
