This is an elegant and brilliant idea and I love it. Finding a way to formalize a baseline and then get notified for deviations from that baseline is a fundamental requirement for having things running smoothly.

I’ve been working on a way to automate daily log parsing to get to the same state but including performance metrics and network traffic would definitely round out the signal.

That article is dated, and a lot of the new improvements specifically offset a lot of the claims being made there which just looking at the tone seem to be more of a hit piece on ollama then an objective look to compare. While llama.cpp does have some minor benefits, a lot of the testing I’ve seen and done shows that they are minimal.

For me at least olama is easy to use, simple to set up, and gets the job done. People can use other tools that’s fine, but as I said, I’ve been using it with no issue for a while now, and when I’ve looked at switching it just doesn’t make any sense.

I’ve had zero issues with ollama, and every day I get closer to no longer using the frontier models.

I may eventually add that, but right now my goal isn’t the actual logging itself, it’s creating a digest and summarizing the logs down to the key points, and eventually getting a notification every morning if something weird showed up. I’m definitely going to add this to my backlog though, thanks for the suggestion

Jev and ollaya are on my list to eventually add. I’m running ollama through an RTX 3080 with 10 gigs of RAM. Just because it’s what I had lying around. I was mostly using it for gaming but it works for local model use. I’m also planning on installing Hermes in a bit once the log manager project is done, but that’s going to take some careful tinkering because I’m technically underpowered for it. I’m probably just going to have some gate checks up to make sure nothing loads during a gaming session.

My main goal is to completely get off the frontier models if possible because I have concerns about the environmental impacts, especially. As a shameless pirate I mind intellectual theft less, but I really do disagree with the environmental impact, especially when a optimized local model plus a bit of scripting and filtering. Does the job 90% of the time

But we’ll see, I’m still tweaking everything, but I think I’m about 90% done with the log normalizer to do most of the heavy lifting act on the logs programmatically, to keep the context window for my local models as optimized as possible.

It’s definitely been a good fun learning project

Oh, one of the first things that I implemented was a hallucination checker. If it cites any data that’s not actively in one of the corpus I’ve supplied, it automatically gets flagged and filtered out. But most of what I do can be handled with internal documentation and material I explicitly provide.

I’ve got grafana and Prometheus, but I’m currently working on a generic log pulling script that will then feed to a local ollama instance to summarize the logs and let me know if there are any potential issues I should be aware of. Grafana is great for current state, but I’m a big believer in watching the logs to see trends, but I’m too lazy to do it by hand.

Once that’s up and running and steady, I’ll probably tie it into ntfy to actually alert me in real time

https://www.idrive.com/pricing this is who I use, 35 bucks a month for 50 TB

Install yes, configure not really.

Is they’re a reason you don’t want to just use tailscale for this? it’s incredibly easy to set up and does exactly what you’re trying to do.