Discussion on Hosting Open WebUI in the Cloud

General
1

I’m up and running with Open WebUI on my local computer and now starting to explore hosting it in the cloud but in a private and secure manner that only I have access to.

The reason I want to do this is so that I can access my digital brain from any device, greater security, and the ability to back up my digital brain so I don’t lose any of my information.

Here is a summary of the video generated with my digital brain:

Here are the primary points from the conversation:

  1. Current Setup and Limitations:

    • The speaker is hosting Open Web UI on a MacBook and can only utilize smaller AI models like PHY3 due to hardware constraints.
    • Drawbacks of local hosting include the lack of backups, expensive hardware requirements for larger models, and difficulty in accessing from other devices.

  2. Cloud Hosting Considerations:

    • The goal is to set up Open Web UI on a cloud hosting provider in a way that is private and secure.
    • DigitalOcean is chosen for hosting, aiming to leverage its cloud infrastructure while maintaining security and privacy.
    • There’s an understanding that using external APIs (e.g., OpenAI or Grok) involves sending information outside the private cloud, contrasting with hosting models locally on DigitalOcean or a personal device.

  3. Local Models and HTTPS Exposure:

    • Whether models hosted on the local laptop can be accessed via Open Web UI on a cloud server is questioned. They can be, but need to be exposed through an HTTPS URL using tools like Ngrok for secure tunneling.

  4. Approach to Deployment:

    • The speaker is considering Google Cloud for deploying AI models, discussing costs, configurations, and cloud provider comparisons.
    • Google Cloud is considered potentially more affordable and transparent in pricing compared to Amazon AWS, which may have hidden costs.
    • There is a discussion about leveraging free credits from Google Cloud ($300) and Amazon AWS tiers, but ultimately deciding to start with Google Cloud due to familiarity and transparency.

  5. Final Decisions:

    • They agree to set up Open Web UI on Google Cloud and use the benefits of free credits despite initial preferences against Google due to its dominant role in AI.
    • Potential future steps might include exploring other cloud providers like AWS, Azure, and DigitalOcean to provide flexible options for hosting.

  6. Technical Requirements and Costs:

    • Discussed the necessary system configurations for running large models like Lama 70B on cloud infrastructure and reviewed different pricing tiers.

By the end of the conversation:

  • The decision was to set up Open Web UI and AI models on Google Cloud to make use of free credits and due to the simplicity of setup as known by the speaker.
  • The approach will provide the ability to host models privately and securely while maintaining control over the data and ensuring robust security measures.
2

Here is a followup discussion between Beck and I discussing my experience using the hosted version of Open WebUI that he setup for me vs. the local instance.

The bottom line is I am currently finding it easier to use the locally hosted system across devices using Ngrok vs. the cloud hosted version.