gemini-pro-vision 看图说话

一、安装

复制代码
   pip install -U langchain-google-vertexai

二、设置访问权限

申请服务账号json格式key

三、完整代码

复制代码
import gradio as gr
import json
import base64
from pathlib import Path
import os
import time
import requests
from fastapi import FastAPI, UploadFile, File
from fastapi.middleware.cors import CORSMiddleware
import uvicorn
from langchain_core.messages import HumanMessage
from langchain_google_vertexai import ChatVertexAI
from langchain_core.output_parsers import StrOutputParser

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "xxx.json"
app = FastAPI()
app.add_middleware(
        CORSMiddleware,
        allow_origins=["*"],
        allow_credentials=True,
        allow_methods=["*"],
        allow_headers=["*"],
    )

def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode("utf-8")

def generate(model, prompt, images_base64):
    llm = ChatVertexAI(model_name=model)
    # example
    message = HumanMessage(
        content=[
            {
                "type": "text",
                "text": prompt,
            },
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{images_base64}"}},
        ]
    )
    parser = StrOutputParser()
    result = llm.invoke([message])
    parserResult = parser.invoke(result)
    return parserResult

def respond(model, image_path, prompt, chat_history):
    print(model, image_path, prompt)
    images_base64 = [encode_image(image_path)]
    bot_message = generate(model, prompt, images_base64)
    chat_history.append((prompt, bot_message))
    time.sleep(1)
    return "", chat_history

with gr.Blocks() as demo:
    gr.Image(value='xxx.png',height=30,width=70, interactive=False, show_download_button=False, show_label=False)
    gr.HTML("""<h1 align="center">图片问答</h1>""")
    
    model = gr.Textbox(value="gemini-pro-vision",label="gemini多模态模型:")
    with gr.Row():
        with gr.Column(scale=1):
            image_path = gr.Image(label="上传图片:",type="filepath", value='Picture1.png')
        with gr.Column(scale=3):
            chatbot = gr.Chatbot()
    prompt = gr.Textbox(label="用户:",value="大童在保险行业的地位如何?使用中文回答。")
    
    clear = gr.ClearButton([prompt, chatbot])
            
    prompt.submit(respond, [model, image_path, prompt, chatbot], [prompt, chatbot])

app = gr.mount_gradio_app(app, demo, path="/")

if __name__ == '__main__':
    uvicorn.run(app='web_gemini:app', host='0.0.0.0', port=8500, workers=1)

四、运行效果

相关推荐
Terio_my20 小时前
Python制作12306查票工具:从零构建铁路购票信息查询系统
开发语言·python·microsoft
万粉变现经纪人20 小时前
如何解决 pip install -r requirements.txt 约束文件 constraints.txt 仅允许固定版本(未锁定报错)问题
开发语言·python·r语言·django·beautifulsoup·pandas·pip
站大爷IP20 小时前
Python定时任务实战:APScheduler从入门到精通
python
Fairy_sevenseven20 小时前
[1]python爬虫入门,爬取豆瓣电影top250实践
开发语言·爬虫·python
ThisIsMirror20 小时前
CompletableFuture并行任务超时处理模板
java·windows·python
java1234_小锋21 小时前
TensorFlow2 Python深度学习 - TensorFlow2框架入门 - 计算图和 tf.function 简介
python·深度学习·tensorflow·tensorflow2
程序员晚枫21 小时前
Python 3.14新特性:Zstandard压缩库正式加入标准库,性能提升30%
python
逆境清醒21 小时前
VS Code配置Python开发环境系列(1)___VScode的安装 ,VScode常用快捷键
vscode·python·visual studio code
万粉变现经纪人21 小时前
如何解决 pip install -r requirements.txt 无效可编辑项 ‘e .‘(-e 拼写错误)问题
开发语言·python·r语言·beautifulsoup·pandas·pip·scipy
潇凝子潇1 天前
在使用Nacos作为注册中心和配置中心时,如何解决服务发现延迟或配置更新不及时的问题
开发语言·python·服务发现