CanRisk AI

CanRIsk-AI: An AI pipline for CanRisk-DB

---

[English](./README.md) | [简体中文](./README_zh.md)

introduction

[CanRisk-DB](https://www.canrisk-ai.com/) is a cancer risk factor database built on AI, while CanRisk-AI is the pipeline for implementing this database. CanRisk-DB is accessible via a web browser. The overall pipeline of CanRisk-AI includes abstract filtering, full-text filtering, and multi-agent information extraction. The entire pipeline is as follows:


install

The code of this project is built on Python, and Python 3.10 or a higher version is required.

Dependencies

• volcengine-python-sdk

pip install -U 'volcengine-python-sdk[ark]'

• pandas
• numpy
• cv2
• jinja2
• lightrag (1.2.7, Please note that LightRAG is updated very frequently, and there are significant differences in interfaces between versions. Newer versions of LightRAG may render the script incompatible)
• nest_asyncio
• openai
• json_repair
• llm2json

step

1. About LLM All large model calls in this project are from Volcengine's platform and are implemented using Ark. For relevant information, please visit [Volcano Ark](https://www.volcengine.com/product/ark).

2. Abstract screening

   • The abstracts in this project are sourced from PubMed, Cochrane, and Embase. The search strategy information will be presented in the supplementary materials after the article is published.
   • The input file is in tsv format and does not require column names. The first column contains unique abstract IDs (which can be defined by the user), and the second column contains the abstract content.

   python 1.Abstract_filter.py input_file.tsv output_file.tsv

3. pdf parsing

   • There are many excellent PDF parsing tools available, such as Dolphin, docling, LlamaParse, and more