Creating a food analysis bot

Nutrition has been important in my entire life. I lost a lot of weight using a mixture of calorie counting and a ketogenic diet in my early 30s. It worked like a charm but was quite some effort to track and get right.

Over the last 10+ years i have regained quite some of the weight due to stress, not moving enough and any other excuse you can think of. Going back to my old ways of losing weight seems so much harder now. Who has time to look out for the right recipes, ingredients, if they are available in your closest supermarket. Do they have the right nutrients al..

So, with the advent of AI I started thinking – there has to be a better, lazier way. I love chats and chatbots so why not start with a telegram bot. You write the bot and it responds like your health assistant, keeping a log, recommending meals, giving you shopping lists. The whole shebang. My personal Star Trek board computer, just need to type. Voice will come later (hopefully?)

Try 1: Python + OpenAI assistants

So i went and wrote a python program that utilizes OpenAI Assistants. I went and gave it information around who I am, what my dietary needs are. Wired it to a telegram bot with some python code and voila.

You can chat with the assistant, it keeps a log of all your food in a nice long context. It worked great for me and my wife. We added the bot to a shared group channel and it gave us everything we need.
It did not log the meals, just gave us nice recommendations and grocery shopping lists.

The best part was telling it to adapt it slightly based on our needs and it updated everything automatically.

However, after a while we ran out of context space, i switched model to Gpt4o and it runs a bit further. This wont hold. What next?

To be honest, the context was getting too long!

Intermezzo: But what about pictures?

On a trip to Greece I had a nice idea: take a picture of my meal and tell me what i ate, including calories.

Turns out, as usual, i wasn’t the first person to have this idea.

I found many apps that promised the same but none on the level of a „out of the box“ Claude model.

Someone’s gotta do something!

Try 2: LangGraph

I talked to my friend Erich, and he mentioned he’d like to participate in making the bot. In parallel i started noticing how much more work this would be to professionalize. So much hard-wired code. No idea about proper running it for multi-users. And the biggest problem: The looming threat of finite context space.

The general idea with LangGraph is quite good. You have agents, tools and a graph based execution engine that glues it all together.
Define some persistence, checkpointers and it does all the magic.

It took us a while to get used to the concepts. How LangGraph wants to be used. The whole library felt clunky and hard to navigate but we ultimately managed to re-write the whole bot to the new framework.
From my point of view there is way too much glue code and internal levers that want to be pushed. However we got it working – barely. All of a sudden the execution times were much longer and there is very little transparency that LangGraph offers – even when using LangSmith. Their own logging platform.

So we added logfire and tried to figure it out.

33 seconds to answer? This should not even take 10 seconds. And there is nothing we can do here.

Try 3: n8n

Visual / Block programming is something of a fad. It keeps bubbling up and every time i use it i am disappointed. Latest case in point: Azure Data Factory.
It’s clunky, it offers very little flexibility – and is usually replaced once the use case gets more complicated.
But still .. i wanted to try. And the choice, after a little bit of research, fell on n8n.

This is how part of the workflow looks like in n8n.

So how is this one different? In 2 days i managed to reproduce the whole bot so far, the execution time to analyse one picture is down to sub 10 seconds.

Here is some pros and cons of my time with n8n.

Conclusion

not yet 🙂