OpenAI开放gpt-3.5turbo微调fine-tuning测试教程

文章目录

openai微调 fine-tuning介绍
openai微调地址

网址:https://platform.openai.com/finetune

jsonl格式数据集准备
  • 使用Chinese-medical-dialogue-data数据集
  • git clone进行下载

git clone https://github.com/Toyhom/Chinese-medical-dialogue-data

  • 选择其中心血管科中的部分数据进行微调

    微调需要进行付费,token越多收费越多,并且gpt-3.5-turbotoken数最多为4096

  • dataframe导入csv文件

python 复制代码
import pandas as pd

df = pd.read_csv('Chinese-medical-dialogue-data/样例_内科5000-6000.csv',encoding='gbk')

df
  • 提取样本
python 复制代码
train_data = df[df['department']=='心血管科'].iloc[0:50,:]
valid_data = df[df['department']=='心血管科'].iloc[50:70,:]

train_data
  • jsonl格式数据构建
python 复制代码
lis1 = []
lis2 = []
sys_content = "You are a specialist in cardiovascular disease and you will apply your expertise to give your specialized answers to patients."

for index,row in train_data.iterrows():
    each = []
    each.append({"role":"system","content":sys_content})
    each.append({"role":"user","content":row['ask']})
    each.append({"role":"assistant","content":row['answer']})
    #print(each)
    lis1.append(each)

for index,row in valid_data.iterrows():
    each = []
    each.append({"role":"system","content":sys_content})
    each.append({"role":"user","content":row['ask']})
    each.append({"role":"assistant","content":row['answer']})
    #print(each)
    lis2.append(each)

lis1
  • jsonl数据导出
python 复制代码
lis1 = []
lis2 = []
sys_content = "You are a specialist in cardiovascular disease and you will apply your expertise to give your specialized answers to patients."

for index,row in train_data.iterrows():
    each = []
    each.append({"role":"system","content":sys_content})
    each.append({"role":"user","content":row['ask']})
    each.append({"role":"assistant","content":row['answer']})
    #print(each)
    lis1.append(each)

for index,row in valid_data.iterrows():
    each = []
    each.append({"role":"system","content":sys_content})
    each.append({"role":"user","content":row['ask']})
    each.append({"role":"assistant","content":row['answer']})
    #print(each)
    lis2.append(each)

lis1
点击上传文件
  • 上传文件(钱不够了)
相关推荐
yyycqupt3 小时前
git使用(一)
git
Kkooe6 小时前
GitLab|数据迁移
运维·服务器·git
Beekeeper&&P...7 小时前
git bash是什么,git是什么,git中的暂存区是什么,git中的本地仓库是什么,git中工作目录指的是什么
开发语言·git·bash
兰琛10 小时前
20241121 android中树结构列表(使用recyclerView实现)
android·gitee
Stara051112 小时前
Git推送+拉去+uwsgi+Nginx服务器部署项目
git·python·mysql·nginx·gitee·github·uwsgi
坐公交也用券12 小时前
使用Python3实现Gitee码云自动化发布
运维·gitee·自动化
lsswear12 小时前
GIT 操作
git
勋勋勋勋小勋勋12 小时前
git分支合并某一次提交
git
PandaCave13 小时前
git常用命令以及注意事项总结
git