{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"authorship_tag":"ABX9TyMOT3pwGy1DlN3A7RsLjrHF"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"code","execution_count":9,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"XCDZW7K5SeiA","executionInfo":{"status":"ok","timestamp":1720006239143,"user_tz":-540,"elapsed":2314,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"6ab2f2b3-d7a9-4c51-c928-0d702dc38280"},"outputs":[{"output_type":"stream","name":"stdout","text":["Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"]}],"source":["# by nitta@tsuda.ac.jp\n","import os\n","is_colab = 'google.colab' in str(get_ipython())   # for Google Colab\n","\n","if is_colab:\n","    from google.colab import drive\n","    drive.mount('/content/drive')\n","    SAVE_PATH='/content/drive/MyDrive/DeepLearning'\n","else:\n","    SAVE_PATH='.'\n","\n","# # variable definition in 'account.py' to access HuggingFace Hub\n","# MyAccount = {\n","#     'HuggingFace': {\n","#         'user': 'YourUserName',\n","#         'token': 'YourTokenWithWritePermission'\n","#     },\n","#     'OpenAI': {\n","#         'api-key': 'YourOpenAI_API_Key'\n","#     },\n","# }\n","ACCOUNT_FILE = os.path.join(SAVE_PATH, 'account.py')\n","%run {ACCOUNT_FILE}             # set 'MyAccount' variable\n","os.environ[\"OPENAI_API_KEY\"] = MyAccount['OpenAI']['api-key']   # by nitta"]},{"cell_type":"markdown","source":["# OpenAI Assistants API\n","# File Search\n","\n","\"File Search\" とは、いろいろな形式のテキストを使って、アシスタントに外部からの知識を拡張できる機能である。\n","\n","公式ドキュメント:\n","<a href=\"https://platform.openai.com/docs/assistants/tools/file-search\">\n","OpenAI Assistant API: File Search\n","</a><br />"],"metadata":{"id":"hqOMdgV6XOXm"}},{"cell_type":"markdown","source":["# File Search のアシスタントを作る手順\n","\n","1. 文書を用意する。\n","1. ベクターストアに追加する\n","1. File Search 用アシスタントを作り、質問する。\n"],"metadata":{"id":"R7UnU4Ed4pTR"}},{"cell_type":"markdown","source":["## 1. 文書を用意する\n","\n","1. なんらかの知識が記述されている文書を用意する。\n","2. 今回は、投資信託における特定のファンドの目論見書を使ってみることにする。\n"," - nissei_nasdaq100_prospectus.pdf\n","<pre>\n","「投資信託説明書（交付目論見書）」\n","(使用開始日 2024.06.21)\n","「＜購入・換金手数料なし＞ニッセイＮＡＳＤＡＱ１００インデックスファンド」\n","</pre>\n","<a href=\"https://search.sbisec.co.jp/v2/popwin/info/connect/fund/2931323300000005.pdf\">\n","https://search.sbisec.co.jp/v2/popwin/info/connect/fund/2931323300000005.pdf\n","</a><br />\n"," - emaxis_slim_sp500_prospectus.pdf\n","<pre>\n","「投資信託説明書（交付目論見書）」\n","(使用開始日 2024.1.25)\n","「eMAXIS Slim 米国株式(S&P500)」\n","</pre>\n","<a href=\"https://search.sbisec.co.jp/v2/popwin/info/connect/fund/0331118700000029.pdf\">\n","https://search.sbisec.co.jp/v2/popwin/info/connect/fund/0331118700000029.pdf\n","</a><br />\n","3. Google Drive 上にpdfファイルを配置する。\n","4. 考えられる質問\n"," - \"主な投資先はどこですか？\"\n"," - \"目的を簡単に教えてください。\"\n"," - \"特色を簡単に教えてください。\"\n"," - \"運用の手数料は１年間でどのくらいですか？\"\n"," - \"組み入れ銘柄のうち、情報技術に関する企業は合計何パーセントですか？\"\n"," - \"ここ5年間の平均の収益率は何パーセントですか？\"\n"],"metadata":{"id":"8D2npRqnj0KF"}},{"cell_type":"code","source":["# pdf file へのパス\n","pdf_path1 = f'{SAVE_PATH}/openai_assistant/data/nissei_nasdaq100_prospectus.pdf'\n","pdf_path2 = f'{SAVE_PATH}/openai_assistant/data/emaxis_slim_sp500_prospectus.pdf'"],"metadata":{"id":"OFCP6NzBsGrR","executionInfo":{"status":"ok","timestamp":1720007207335,"user_tz":-540,"elapsed":312,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":24,"outputs":[]},{"cell_type":"markdown","source":["## 2. 前準備"],"metadata":{"id":"WlxlSlD9fy6O"}},{"cell_type":"code","source":["# パッケージのインストール\n","!pip install openai"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"uwGkNvfeTHc0","executionInfo":{"status":"ok","timestamp":1719999872607,"user_tz":-540,"elapsed":7754,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"e4d034b6-845a-439a-985e-dc0f2d9c5b14"},"execution_count":6,"outputs":[{"output_type":"stream","name":"stdout","text":["Requirement already satisfied: openai in /usr/local/lib/python3.10/dist-packages (1.35.9)\n","Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai) (3.7.1)\n","Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai) (1.7.0)\n","Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from openai) (0.27.0)\n","Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai) (2.7.4)\n","Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai) (1.3.1)\n","Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai) (4.66.4)\n","Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai) (4.12.2)\n","Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (3.7)\n","Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai) (1.2.1)\n","Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (2024.6.2)\n","Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (1.0.5)\n","Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai) (0.14.0)\n","Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai) (0.7.0)\n","Requirement already satisfied: pydantic-core==2.18.4 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai) (2.18.4)\n"]}]},{"cell_type":"code","source":["from openai import OpenAI\n","\n","# クライアントの準備\n","client = OpenAI()"],"metadata":{"id":"j13bz-6yf7dD","executionInfo":{"status":"ok","timestamp":1719999872607,"user_tz":-540,"elapsed":3,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":7,"outputs":[]},{"cell_type":"code","source":["import json\n","\n","# JSON出力ヘルパーの準備\n","def show_json(obj):\n","    display(json.loads(obj.model_dump_json()))"],"metadata":{"id":"PHyYTUpwf9RE","executionInfo":{"status":"ok","timestamp":1719999872607,"user_tz":-540,"elapsed":2,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":8,"outputs":[]},{"cell_type":"markdown","source":["## 3. アシスタントを作成する\n","\n","<code>tools</code>\n","パラメータとして\n","<code>file_search</code>\n","を渡して、アシスタントを作成する。"],"metadata":{"id":"IOsnwWatgBLa"}},{"cell_type":"code","source":["assistant = client.beta.assistants.create(\n","    name = \"Finanicial Analyst Assistant\",\n","    instructions = \"あなたは株式投資の専門家です。知識ベースを利用して、投資信託の目論見書に関する質問に回答してください。\",\n","    model=\"gpt-4o\",\n","    tools=[{'type': 'file_search'}],\n",")"],"metadata":{"id":"hwNawYcff_pa","executionInfo":{"status":"ok","timestamp":1720006420896,"user_tz":-540,"elapsed":853,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":11,"outputs":[]},{"cell_type":"markdown","source":["## 4. ファイルをアップロードして、ベクターストアに追加する\n","\n","追加したファイルにアクセスするために、<code>file_search</code> ツールは VectorStore を使う。\n","\n","<code>vector_stores.file.batches.upload_and_poll()</code> は、\n","ファイルをアップロードした後、\n","すべてのコンテンツの処理が終了するのを待ってから値を返す。"],"metadata":{"id":"1jXr5jPLYfaB"}},{"cell_type":"code","source":["! ls -l {pdf_path1}"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"RcaTlVbiaDcE","executionInfo":{"status":"ok","timestamp":1720007228968,"user_tz":-540,"elapsed":302,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"8ad193a2-0976-4c2b-cbad-39678f35b647"},"execution_count":26,"outputs":[{"output_type":"stream","name":"stdout","text":["-rw------- 1 root root 750721 Jul  3 07:55 /content/drive/MyDrive/DeepLearning/openai_assistant/data/nissei_nasdaq100_prospectus.pdf\n"]}]},{"cell_type":"code","source":["# pdf file を upload する。\n","vector_store = client.beta.vector_stores.create(name=\"Investment Trust Prospectus\")\n","\n","file_paths = [pdf_path1, pdf_path2]\n","file_streams = [ open(path, \"rb\") for path in file_paths ]\n","\n","file_batch = client.beta.vector_stores.file_batches.upload_and_poll(\n","    vector_store_id=vector_store.id,\n","    files=file_streams\n",")"],"metadata":{"id":"qnIt5pwgXut-","executionInfo":{"status":"ok","timestamp":1720007239720,"user_tz":-540,"elapsed":5305,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":27,"outputs":[]},{"cell_type":"code","source":["show_json(vector_store)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":277},"id":"FkMKzrUbbM9Q","executionInfo":{"status":"ok","timestamp":1720007326917,"user_tz":-540,"elapsed":327,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"696ed673-035b-42cd-d4bb-845b660000a0"},"execution_count":31,"outputs":[{"output_type":"display_data","data":{"text/plain":["{'id': 'vs_MZsrj644te0JhiMJDIsxu4p7',\n"," 'created_at': 1720007235,\n"," 'file_counts': {'cancelled': 0,\n","  'completed': 0,\n","  'failed': 0,\n","  'in_progress': 0,\n","  'total': 0},\n"," 'last_active_at': 1720007235,\n"," 'metadata': {},\n"," 'name': 'Investment Trust Prospectus',\n"," 'object': 'vector_store',\n"," 'status': 'completed',\n"," 'usage_bytes': 0,\n"," 'expires_after': None,\n"," 'expires_at': None}"]},"metadata":{}}]},{"cell_type":"code","source":["show_json(file_batch)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":191},"id":"SwnOwwdbZ5zg","executionInfo":{"status":"ok","timestamp":1720007275084,"user_tz":-540,"elapsed":378,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"84af536e-c066-4f05-f9ea-8b998ead88cc"},"execution_count":29,"outputs":[{"output_type":"display_data","data":{"text/plain":["{'id': 'vsfb_0bc90e5959174c2e975e16d10e6ac2ec',\n"," 'created_at': 1720007240,\n"," 'file_counts': {'cancelled': 0,\n","  'completed': 2,\n","  'failed': 0,\n","  'in_progress': 0,\n","  'total': 2},\n"," 'object': 'vector_store.file_batch',\n"," 'status': 'completed',\n"," 'vector_store_id': 'vs_MZsrj644te0JhiMJDIsxu4p7'}"]},"metadata":{}}]},{"cell_type":"code","source":["print(file_batch.status)\n","print(file_batch.file_counts)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"2_zsSdTAbBIa","executionInfo":{"status":"ok","timestamp":1720007289434,"user_tz":-540,"elapsed":318,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"17b8f58d-194d-47d2-ac36-d05b2563f2c7"},"execution_count":30,"outputs":[{"output_type":"stream","name":"stdout","text":["completed\n","FileCounts(cancelled=0, completed=2, failed=0, in_progress=0, total=2)\n"]}]},{"cell_type":"markdown","source":["## 5. 新しい VectorStore を使うためにアシスタントを更新する\n","\n","アシスタントが追加したファイルにアクセスできるように、\n","アシスタントの <code>tool_resources</code> を\n","新しい VectorStore の ID で更新する。\n"],"metadata":{"id":"DzE-og0SbdvY"}},{"cell_type":"code","source":["assistant = client.beta.assistants.update(\n","    assistant_id = assistant.id,\n","    tool_resources={ 'file_search': { 'vector_store_ids': [vector_store.id]}}\n",")"],"metadata":{"id":"La2ZK-yMfntB","executionInfo":{"status":"ok","timestamp":1720008538155,"user_tz":-540,"elapsed":781,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":34,"outputs":[]},{"cell_type":"markdown","source":["## 6. スレッドを作成する\n","\n","スレッドに Message Attatchment としてファイルを添付することもできる。\n","これを行うと、新しい VectorStore が作成されて、スレッドに添付される。\n","既にスレッドに VectorStore が添付されている場合は、新しいファイルを既存の\n","スレッド VectorStore に添付する。\n","\n","このスレッドで Run を作成すると、\n","file search ツールはアシスタントのVectorStoreとスレッドのVectorStore の両方を探索する。\n","\n"],"metadata":{"id":"i1uh3Q3Hf4mL"}},{"cell_type":"code","source":["message_file = client.files.create(\n","    file=open(pdf_path1, \"rb\"),\n","    purpose='assistants',\n",")"],"metadata":{"id":"QK5RaTEobGdq","executionInfo":{"status":"ok","timestamp":1720009347536,"user_tz":-540,"elapsed":807,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":36,"outputs":[]},{"cell_type":"code","source":["thread = client.beta.threads.create(\n","    messages = [\n","        {\n","            'role': 'user',\n","            'content': 'nasdaq100 の特色を教えてください。',\n","            'attachments': [\n","                {\n","                    'file_id': message_file.id,\n","                    'tools': [ { 'type': 'file_search' }]\n","                }\n","            ],\n","        }\n","    ]\n",")"],"metadata":{"id":"BNzZr2Wzi5nk","executionInfo":{"status":"ok","timestamp":1720009557642,"user_tz":-540,"elapsed":787,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":38,"outputs":[]},{"cell_type":"code","source":["print(thread.tool_resources.file_search)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"1t2Hf5OljkRF","executionInfo":{"status":"ok","timestamp":1720009576169,"user_tz":-540,"elapsed":313,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"3fc479b1-1c4e-47f6-852e-97a17f689354"},"execution_count":39,"outputs":[{"output_type":"stream","name":"stdout","text":["ToolResourcesFileSearch(vector_store_ids=['vs_0n2Z8YzEVZkw9P3u1oIqiwFf'])\n"]}]},{"cell_type":"markdown","source":["## 7. Run を作成する。\n","\n","\"With Streaming\" と \"Without Streaming\" の2通りがある。\n","\n","ここでは、簡単のため \"Without Streaming\" の場合を述べる。"],"metadata":{"id":"-R24r8C0kHdf"}},{"cell_type":"code","source":["# Run を作成する。\n","run = client.beta.threads.runs.create_and_poll(\n","    thread_id = thread.id,\n","    assistant_id = assistant.id,\n",")"],"metadata":{"id":"G1E2EH_Pj0wE","executionInfo":{"status":"ok","timestamp":1720009804454,"user_tz":-540,"elapsed":11294,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":40,"outputs":[]},{"cell_type":"code","source":["print(run)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"rrGXO3NXkvkc","executionInfo":{"status":"ok","timestamp":1720009821655,"user_tz":-540,"elapsed":346,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"f4837f0b-2b73-4e4a-a375-c82ecfd0d5be"},"execution_count":41,"outputs":[{"output_type":"stream","name":"stdout","text":["Run(id='run_DzROST0kWMrFSvVzVmGLvivh', assistant_id='asst_Y4FHrpxaGJDHjb8yajurHqiU', cancelled_at=None, completed_at=1720009805, created_at=1720009794, expires_at=None, failed_at=None, incomplete_details=None, instructions='あなたは株式投資の専門家です。知識ベースを利用して、投資信託の目論見書に関する質問に回答してください。', last_error=None, max_completion_tokens=None, max_prompt_tokens=None, metadata={}, model='gpt-4o', object='thread.run', parallel_tool_calls=True, required_action=None, response_format='auto', started_at=1720009794, status='completed', thread_id='thread_eKcaCt68A5Dhxrka2mqCzOwJ', tool_choice='auto', tools=[FileSearchTool(type='file_search', file_search=None)], truncation_strategy=TruncationStrategy(type='auto', last_messages=None), usage=Usage(completion_tokens=371, prompt_tokens=12799, total_tokens=13170), temperature=1.0, top_p=1.0, tool_resources={})\n"]}]},{"cell_type":"code","source":["show_json(run)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":538},"id":"jAqxVmj8kvwM","executionInfo":{"status":"ok","timestamp":1720009830295,"user_tz":-540,"elapsed":322,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"90770b58-99f2-4b73-fa52-a68aa8c3c623"},"execution_count":42,"outputs":[{"output_type":"display_data","data":{"text/plain":["{'id': 'run_DzROST0kWMrFSvVzVmGLvivh',\n"," 'assistant_id': 'asst_Y4FHrpxaGJDHjb8yajurHqiU',\n"," 'cancelled_at': None,\n"," 'completed_at': 1720009805,\n"," 'created_at': 1720009794,\n"," 'expires_at': None,\n"," 'failed_at': None,\n"," 'incomplete_details': None,\n"," 'instructions': 'あなたは株式投資の専門家です。知識ベースを利用して、投資信託の目論見書に関する質問に回答してください。',\n"," 'last_error': None,\n"," 'max_completion_tokens': None,\n"," 'max_prompt_tokens': None,\n"," 'metadata': {},\n"," 'model': 'gpt-4o',\n"," 'object': 'thread.run',\n"," 'parallel_tool_calls': True,\n"," 'required_action': None,\n"," 'response_format': 'auto',\n"," 'started_at': 1720009794,\n"," 'status': 'completed',\n"," 'thread_id': 'thread_eKcaCt68A5Dhxrka2mqCzOwJ',\n"," 'tool_choice': 'auto',\n"," 'tools': [{'type': 'file_search', 'file_search': None}],\n"," 'truncation_strategy': {'type': 'auto', 'last_messages': None},\n"," 'usage': {'completion_tokens': 371,\n","  'prompt_tokens': 12799,\n","  'total_tokens': 13170},\n"," 'temperature': 1.0,\n"," 'top_p': 1.0,\n"," 'tool_resources': {}}"]},"metadata":{}}]},{"cell_type":"markdown","source":["## 8. 出力を確認する\n"],"metadata":{"id":"bgTMzPcZk27B"}},{"cell_type":"code","source":["messages = list(client.beta.threads.messages.list(thread_id=thread.id, run_id=run.id))"],"metadata":{"id":"va1OI_sPkyy3","executionInfo":{"status":"ok","timestamp":1720009920574,"user_tz":-540,"elapsed":805,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":44,"outputs":[]},{"cell_type":"code","source":["print(messages)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"inZ0btNclCQ7","executionInfo":{"status":"ok","timestamp":1720009927876,"user_tz":-540,"elapsed":338,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"f4c66fe9-ca8f-484e-ed78-eddc9310cc53"},"execution_count":45,"outputs":[{"output_type":"stream","name":"stdout","text":["[Message(id='msg_Uqg4Ky9Ed6TJFR8c7Ywxbws0', assistant_id='asst_Y4FHrpxaGJDHjb8yajurHqiU', attachments=[], completed_at=None, content=[TextContentBlock(text=Text(annotations=[FileCitationAnnotation(end_index=123, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=111, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=135, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=123, text='【4:2†source】', type='file_citation'), FileCitationAnnotation(end_index=212, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=200, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=224, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=212, text='【4:1†source】', type='file_citation'), FileCitationAnnotation(end_index=280, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=268, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=292, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=280, text='【4:2†source】', type='file_citation'), FileCitationAnnotation(end_index=361, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=349, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=373, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=361, text='【4:2†source】', type='file_citation')], value='NASDAQ100の特色について、以下のポイントが挙げられます：\\n\\n1. **インデックス構成**: \\n   NASDAQ100指数は、NASDAQ市場に上場している金融銘柄を除いた時価総額上位100銘柄で構成されています【4:0†source】【4:2†source】。\\n\\n2. **投資の目標**: \\n   NASDAQ100指数（配当込み、円換算ベース）の動きに連動する投資成果を目指しています【4:0†source】【4:1†source】。\\n\\n3. **為替ヘッジの方針**:\\n   原則として、対円での為替ヘッジは行いません【4:0†source】【4:2†source】。\\n\\n4. **投資対象**:\\n   DR（預託証券）や株式等と同等の投資効果が得られる証券および証書等を含みます【4:0†source】【4:2†source】。\\n\\nNASADAQ100は、特にハイテク企業が多く含まれており、技術革新の動向による影響を受けやすいインデックスです。また、為替リスクや各国の政治・経済情勢などにも影響を受ける点が留意点となります。\\n\\nこれらの特色は、投資判断をする上で重要な情報となりますので、リスク管理を含めて総合的に考えられることが望ましいです。'), type='text')], created_at=1720009798, incomplete_at=None, incomplete_details=None, metadata={}, object='thread.message', role='assistant', run_id='run_DzROST0kWMrFSvVzVmGLvivh', status=None, thread_id='thread_eKcaCt68A5Dhxrka2mqCzOwJ')]\n"]}]},{"cell_type":"code","source":["print(len(messages))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ab7iSCyglKnc","executionInfo":{"status":"ok","timestamp":1720009952185,"user_tz":-540,"elapsed":303,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"f07e5281-2cc1-421b-eee1-78432b72dcc9"},"execution_count":47,"outputs":[{"output_type":"stream","name":"stdout","text":["1\n"]}]},{"cell_type":"code","source":["print(messages[0])"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"pX8J96JqlVet","executionInfo":{"status":"ok","timestamp":1720009978094,"user_tz":-540,"elapsed":300,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"ff2e66a3-a39b-4276-83cb-7a1c195ad426"},"execution_count":48,"outputs":[{"output_type":"stream","name":"stdout","text":["Message(id='msg_Uqg4Ky9Ed6TJFR8c7Ywxbws0', assistant_id='asst_Y4FHrpxaGJDHjb8yajurHqiU', attachments=[], completed_at=None, content=[TextContentBlock(text=Text(annotations=[FileCitationAnnotation(end_index=123, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=111, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=135, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=123, text='【4:2†source】', type='file_citation'), FileCitationAnnotation(end_index=212, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=200, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=224, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=212, text='【4:1†source】', type='file_citation'), FileCitationAnnotation(end_index=280, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=268, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=292, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=280, text='【4:2†source】', type='file_citation'), FileCitationAnnotation(end_index=361, file_citation=FileCitation(file_id='file-P7iORP5ju1WpgTWTFA8CpL64'), start_index=349, text='【4:0†source】', type='file_citation'), FileCitationAnnotation(end_index=373, file_citation=FileCitation(file_id='file-drSzFl5teC9JRYQ5Et5JpnPc'), start_index=361, text='【4:2†source】', type='file_citation')], value='NASDAQ100の特色について、以下のポイントが挙げられます：\\n\\n1. **インデックス構成**: \\n   NASDAQ100指数は、NASDAQ市場に上場している金融銘柄を除いた時価総額上位100銘柄で構成されています【4:0†source】【4:2†source】。\\n\\n2. **投資の目標**: \\n   NASDAQ100指数（配当込み、円換算ベース）の動きに連動する投資成果を目指しています【4:0†source】【4:1†source】。\\n\\n3. **為替ヘッジの方針**:\\n   原則として、対円での為替ヘッジは行いません【4:0†source】【4:2†source】。\\n\\n4. **投資対象**:\\n   DR（預託証券）や株式等と同等の投資効果が得られる証券および証書等を含みます【4:0†source】【4:2†source】。\\n\\nNASADAQ100は、特にハイテク企業が多く含まれており、技術革新の動向による影響を受けやすいインデックスです。また、為替リスクや各国の政治・経済情勢などにも影響を受ける点が留意点となります。\\n\\nこれらの特色は、投資判断をする上で重要な情報となりますので、リスク管理を含めて総合的に考えられることが望ましいです。'), type='text')], created_at=1720009798, incomplete_at=None, incomplete_details=None, metadata={}, object='thread.message', role='assistant', run_id='run_DzROST0kWMrFSvVzVmGLvivh', status=None, thread_id='thread_eKcaCt68A5Dhxrka2mqCzOwJ')\n"]}]},{"cell_type":"code","source":["show_json(messages[0])"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"kSaTwWKqla3T","executionInfo":{"status":"ok","timestamp":1720010003337,"user_tz":-540,"elapsed":4,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"b3b12d88-9748-4bac-914a-44a4f4e62539"},"execution_count":49,"outputs":[{"output_type":"display_data","data":{"text/plain":["{'id': 'msg_Uqg4Ky9Ed6TJFR8c7Ywxbws0',\n"," 'assistant_id': 'asst_Y4FHrpxaGJDHjb8yajurHqiU',\n"," 'attachments': [],\n"," 'completed_at': None,\n"," 'content': [{'text': {'annotations': [{'end_index': 123,\n","      'file_citation': {'file_id': 'file-P7iORP5ju1WpgTWTFA8CpL64'},\n","      'start_index': 111,\n","      'text': '【4:0†source】',\n","      'type': 'file_citation'},\n","     {'end_index': 135,\n","      'file_citation': {'file_id': 'file-drSzFl5teC9JRYQ5Et5JpnPc'},\n","      'start_index': 123,\n","      'text': '【4:2†source】',\n","      'type': 'file_citation'},\n","     {'end_index': 212,\n","      'file_citation': {'file_id': 'file-P7iORP5ju1WpgTWTFA8CpL64'},\n","      'start_index': 200,\n","      'text': '【4:0†source】',\n","      'type': 'file_citation'},\n","     {'end_index': 224,\n","      'file_citation': {'file_id': 'file-drSzFl5teC9JRYQ5Et5JpnPc'},\n","      'start_index': 212,\n","      'text': '【4:1†source】',\n","      'type': 'file_citation'},\n","     {'end_index': 280,\n","      'file_citation': {'file_id': 'file-P7iORP5ju1WpgTWTFA8CpL64'},\n","      'start_index': 268,\n","      'text': '【4:0†source】',\n","      'type': 'file_citation'},\n","     {'end_index': 292,\n","      'file_citation': {'file_id': 'file-drSzFl5teC9JRYQ5Et5JpnPc'},\n","      'start_index': 280,\n","      'text': '【4:2†source】',\n","      'type': 'file_citation'},\n","     {'end_index': 361,\n","      'file_citation': {'file_id': 'file-P7iORP5ju1WpgTWTFA8CpL64'},\n","      'start_index': 349,\n","      'text': '【4:0†source】',\n","      'type': 'file_citation'},\n","     {'end_index': 373,\n","      'file_citation': {'file_id': 'file-drSzFl5teC9JRYQ5Et5JpnPc'},\n","      'start_index': 361,\n","      'text': '【4:2†source】',\n","      'type': 'file_citation'}],\n","    'value': 'NASDAQ100の特色について、以下のポイントが挙げられます：\\n\\n1. **インデックス構成**: \\n   NASDAQ100指数は、NASDAQ市場に上場している金融銘柄を除いた時価総額上位100銘柄で構成されています【4:0†source】【4:2†source】。\\n\\n2. **投資の目標**: \\n   NASDAQ100指数（配当込み、円換算ベース）の動きに連動する投資成果を目指しています【4:0†source】【4:1†source】。\\n\\n3. **為替ヘッジの方針**:\\n   原則として、対円での為替ヘッジは行いません【4:0†source】【4:2†source】。\\n\\n4. **投資対象**:\\n   DR（預託証券）や株式等と同等の投資効果が得られる証券および証書等を含みます【4:0†source】【4:2†source】。\\n\\nNASADAQ100は、特にハイテク企業が多く含まれており、技術革新の動向による影響を受けやすいインデックスです。また、為替リスクや各国の政治・経済情勢などにも影響を受ける点が留意点となります。\\n\\nこれらの特色は、投資判断をする上で重要な情報となりますので、リスク管理を含めて総合的に考えられることが望ましいです。'},\n","   'type': 'text'}],\n"," 'created_at': 1720009798,\n"," 'incomplete_at': None,\n"," 'incomplete_details': None,\n"," 'metadata': {},\n"," 'object': 'thread.message',\n"," 'role': 'assistant',\n"," 'run_id': 'run_DzROST0kWMrFSvVzVmGLvivh',\n"," 'status': None,\n"," 'thread_id': 'thread_eKcaCt68A5Dhxrka2mqCzOwJ'}"]},"metadata":{}}]},{"cell_type":"code","source":["message_content = messages[0].content[0].text\n","annotations = message_content.annotations\n","citations = []\n","for index, annotation in enumerate(annotations):\n","    message_content.value = message_content.value.replace(annotation.text, f\"[{index}]\")\n","    if file_citation := getattr(annotation, \"file_citation\", None):\n","        cited_file = client.files.retrieve(file_citation.file_id)\n","        citations.append(f'[{index}] {cited_file.filename}')"],"metadata":{"id":"2R6yMSq0lMPO","executionInfo":{"status":"ok","timestamp":1720010248843,"user_tz":-540,"elapsed":2050,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}}},"execution_count":53,"outputs":[]},{"cell_type":"code","source":["print(message_content.value)\n","print('\\n'.join(citations))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Insgg48Olq5_","executionInfo":{"status":"ok","timestamp":1720010267813,"user_tz":-540,"elapsed":2,"user":{"displayName":"Yoshihisa Nitta","userId":"15888006800030996813"}},"outputId":"5701cae1-ac35-4a71-abbb-6dea9cf9dc57"},"execution_count":54,"outputs":[{"output_type":"stream","name":"stdout","text":["NASDAQ100の特色について、以下のポイントが挙げられます：\n","\n","1. **インデックス構成**: \n","   NASDAQ100指数は、NASDAQ市場に上場している金融銘柄を除いた時価総額上位100銘柄で構成されています[0][1]。\n","\n","2. **投資の目標**: \n","   NASDAQ100指数（配当込み、円換算ベース）の動きに連動する投資成果を目指しています[0][3]。\n","\n","3. **為替ヘッジの方針**:\n","   原則として、対円での為替ヘッジは行いません[0][1]。\n","\n","4. **投資対象**:\n","   DR（預託証券）や株式等と同等の投資効果が得られる証券および証書等を含みます[0][1]。\n","\n","NASADAQ100は、特にハイテク企業が多く含まれており、技術革新の動向による影響を受けやすいインデックスです。また、為替リスクや各国の政治・経済情勢などにも影響を受ける点が留意点となります。\n","\n","これらの特色は、投資判断をする上で重要な情報となりますので、リスク管理を含めて総合的に考えられることが望ましいです。\n","[0] nissei_nasdaq100_prospectus.pdf\n","[1] nissei_nasdaq100_prospectus.pdf\n","[2] nissei_nasdaq100_prospectus.pdf\n","[3] nissei_nasdaq100_prospectus.pdf\n","[4] nissei_nasdaq100_prospectus.pdf\n","[5] nissei_nasdaq100_prospectus.pdf\n","[6] nissei_nasdaq100_prospectus.pdf\n","[7] nissei_nasdaq100_prospectus.pdf\n"]}]},{"cell_type":"code","source":[],"metadata":{"id":"LMhykW2_mdpt"},"execution_count":null,"outputs":[]}]}