The Graph Layer Behind NASA’s Breakthroughs

[00:00:00] Simon: Hello and welcome to another episode of the AI Native Dev, and joining me today is a great old friend from, uh, from many, many years ago.

[00:00:17] Michael: Yeah, that's true.

[00:00:18] Simon: We, we were talking about 15 years or so. Yeah. Michael Hunger, developer advocate, head of developer advocacy at…

[00:00:24] Michael: Actually, I moved to product now,

Simon: Oh, I’m sorry.

[00:00:27] Michael: VP of Product Innovation.

Simon: Wow.

Michael: So, I'm doing all the cool new stuff. I can basically see how you integrate all the modern developments, especially GenAI today, but also, like, uh, cloud integrations and so on into the graph database.

Simon: Congratulations, that’s awesome.

Michael: It's basically what I've been doing, even in DevRev for a long time, with Neo4j Labs and so on.

[00:00:47] Michael: But now, like officially in, in the product work and, and driving a lot of these innovation topics.

Simon: Oh, oh, that's great. That's great.

Michael: It's a lot of fun.

[00:00:55] Simon: Well that goes to show we didn't meet today, right?

Michael: Exactly. Yeah.

[00:01:00] Simon: I'm still here at AI-Fokus here in Stockholm. And yeah, I bumped into Michael both in the corridor and also in the keynote.

[00:01:09] Simon: Um, you've been at Neo4j for, like I say, 15 years or so as well. Um, tell us a little bit about Neo4j because this is, it's a very interesting, kinda like technology that really shows graph databases and what graph databases are.

[00:01:24] Michael: Yeah. So graph databases, many people basically know as relational databases.

[00:01:28] Michael: They have rows and columns and tables. But in a graph database, you have nodes and relationships. Or entities and relationships. Basically what you would draw on a whiteboard, right?

So you have circles and arrows between them, and then you can put arbitrary information on the entities, but also on a relationship, which is actually quite important.

[00:01:43] Michael: And you connect information, right? So you connect people, events, processes, products, whatever, right? So you can do traditional use cases like supply chain, software analytics, cybersecurity, and other things. Um, and what's actually especially interesting is that, unlike in relation to other databases that compute connections between information at query time, in a graph database we persist it at insertion time. So we basically materialize these relationships as first-class citizens in the database. And then when it comes to querying, we just follow pointers in the data, which makes it much, much faster for these complex queries as such. So Neo4j has been around for a long time, it’s been an open source database for a long time.

[00:02:21] Michael: It has Swedish roots as well.

Simon: Really?

Michael: So they come from Sweden actually. And, uh, so, and now we've of course expanded globally, but I joined when we were 10 people.

Simon: Wow!

Michael: Like seven Swedes, a Brit, an American and me.

[00:02:34] Simon: Really? That sounds like the start of a joke, doesn't it?

Michael: Yeah. Like seven Swedes, a Brit, an American and me going to a bar.

[00:02:42] Simon: And all of a sudden they form relationships and graph databases. There we go. So, yeah, I expect no one laughed at that in the, on the, on the audience just then, but we laughed, so that's the important thing. So, yeah, graph databases have been around a long time.

[00:02:58] Simon: People who care about the relationships, yeah, between nodes, the relationships between data, which is very, very hard to essentially, almost like you're forcing it in a relational, in a relational data.

[00:03:08] Michael: Yeah. It's quite a pain. If you, if you tried this ever, I mean, you know, object relation database…

[00:03:12] Michael: impedance mismatch, right? So that's like even, that's even a term for it. Whenever you try to take a rich domain model and shoehorn it into relation, it's always painful, right?

Simon: Yes.

Michael: So, and because you’re in a graph, it's more natural. And it's also what's interesting, you don't atleast lose the business people because in a relational database, if I show a business person like 250 tables, there's a, okay, what do you want to tell me?

[00:03:31] Michael: Yeah. But if you show them basically connected information on the screen, how all the things go together, then say,“Oh, I see!,” and you match this relationship here.

[00:03:39] Simon: Yeah.

[00:03:39] Michael: By the way, something like that. So you also take the people from the business side along, yeah, which makes for a much richer development experience.

[00:03:48] Simon: It's interesting, it's interesting actually, because it actually carries intent, doesn't it? The relationships show if you care about the relationship between two nodes. That's an intent—that is something that is important to the business.

Michael: Exactly.

Simon: Okay. You care about the business.

[00:03:58] Michael: And because you can put attributes on relationships as well.

[00:04:00] Michael: You can actually score this intent. You can even put weights and, and, and scores and things like that on that as well.

Simon: Interesting. Yeah.

Michael: Or like timeframes. Right? So if you have certain relationships that only exist in a certain timeframe, then you can basically put this information on it as well, which you then can use for accommodation, for fraud detection and other things as well.

[00:04:16] Simon: Yeah.

[00:04:16] Michael: Not just the attributes on the entities are important, but also how are they connected and when and how strongly are they also connected?

Simon: Yeah. Yeah. Cool.

Michael: Which then ties actually even further into graph algorithms and stuff like that. They go into page range and clustering and like all the other things.

[00:04:29] Simon: To see how close nodes are and things like that.

[00:04:31] Michael: Exactly. Or how important one node is in a network. For instance, if you have something like a fraudster and they have a really high centrality, that means they had a lot of, already, influence in this. But if they have lower scores, uh, uh, centrality scores, then they have not started their fraud business yet.

[00:04:47] Michael: So you can catch them before the effect basically.

[00:04:49] Simon: Yeah. Yeah. Oh, cool. And, of course graph databases have been around for a while.

Michael: For a while.

[00:04:55] Simon: Uh, they're not new. They're not like, you know…

[00:04:58] Michael: I mean, I still love it and I'm still excited about it.

Simon: Yeah, yeah.

Michael: But, you know, we've been around for a while and people have been using Neo4j for a long time.

[00:05:05] Michael: Yeah.

[00:05:05] Simon: And just in their normal applications where the relationships between data is important, graph databases are a great way to implement that and then use that within your application. However, now AI has come along,

Michael: Everything is changing.

Simon: Um, everything changes. And now I mentioned you got, you've got two sessions.

[00:05:20] Simon: One was about how, um, graph databases can help you, particularly in and around the RAG space, actually pulling context in whereby if you, you know, get some kind of a prompt, you can actually take—you can actually understand relationships between what you know, what you're hitting in your RAG database equivalent.

[00:05:40] Simon: Um, and as a result, you can actually pull various bits of context that are actually, uh, adjacently important.

[00:05:48] Michael: Exactly. The adjacency, right? Yeah. And the surroundings are important, right? Not just the exact, like, thing that you get back from your vector search or something like that, but actually what's surrounding it.

[00:05:55] Michael: How does it connect to the context of the user who's actually asking the question, right? How does it connect to the business, uh, processes and other things, right? Yeah. So you get a much richer answer back from the database, which you then pass to the other LLM for the final answer.

[00:06:08] Simon: Amazing. So I'd love to talk about that.

[00:06:08] Simon: And then I'll, I'll I, and then you, then you go on later this afternoon to talk about MCP.

[00:06:14] Michael: My other favorite one, one of my other favorite topics.

[00:06:15] Simon: And specifically about how devs and dev tools can be used with MCP to effectively make developers more productive. So we really wanna focus on that as well.

[00:06:22] Simon: But let's talk about, let's talk about how you can use graph databases in RAG. I guess to start off with, when we use, when we, we kinda like, you know, use more traditional, kinda like not traditional, you're only a few years in, right?

Michael: Yeah.

Simon: But you know, non-graph based RAG. Basic RAG.

[00:06:37] Simon: Yeah. Um. What are, I think most people know how, kinda like, you know, people use RAG today. What are the, what are the downfalls there? What are the, what are the pitfalls that actually say, do you know what, this isn't, this isn't good enough. Yeah. We need to be better.

[00:06:51] Michael: Yeah. Actually, it's kind of interesting.

[00:06:53] Michael: So, uh, I was at an AI engineering, uh, conference yesterday, uh, last week in

Simon: San Francisco.

Michael: San Francisco, and they actually put up an RAG assistant, uh, using vector search of their conference.

Simon: Oh, right.

Michael: And I tried to find my talk, putting in my name, it didn't find anything. I was talking to a bunch of people from Microsoft about MCP and they had a talk as well.

[00:07:12] Michael: And I was like, show me all the talks from Microsoft and MCP, and it didn't show anything, right? So it did show me two other talks, but not the one that I was actually looking for.

[00:07:20] Michael: So what I want to say is the vector search of content, where you only have fragments of—or content has a bunch of downfalls. One is a black box, right?

[00:07:28] Michael: Yeah. So you basically do vector search, but you don't know why and how you actually get these responses. The other thing is also, it doesn't take context into account, right? So it doesn't take into account like, like MCP is actually a topic and this topic is connected to other talks, right? Yeah. So even if it didn't find a talk in the first…

[00:07:42] Michael: vector search, it could actually, via the context of MCP, pull the other talks in, for instance, right? Yeah. So, and then, uh, from that perspective, that's something where if you only limit yourself to the text fragments, you are losing out on a lot of information that's actually in your data, right? Yeah. So most people actually have more than documents.

[00:07:57] Michael: They have already structured data. They have semistructured data and lots of other data sources.

[00:08:02] Simon: And what was interesting actually, one of the examples you gave in the keynote was, so I think it was about similar to a company that has, maybe it was putting in a, a, um, it had an issue or something like that.

[00:08:12] Simon: And it was a ticket that was raised.

Michael: Yeah.

Simon: Other things like, well, what other tickets have been raised on this same topic? What similar tickets has the company raised? And you can all, you can all, all of a sudden, based on a request, start getting a huge amount of context, not just about the text string, but about, about the individual, the company, the, the, the relationship.

[00:08:32] Michael: Exactly. The customer.

Simon: The customer. Yeah.

[00:08:34] Michael: Look at the customer, for instance, from, from the input. You don't just look at the question string, but you also look at the customers asking it, and you actually look at what has the, what has the customer actually deployed and what is enabled and, and, and so on.

[00:08:46] Michael: And then you tie it back to the, to the support case and say, okay, have they already asked previously? Yeah, similar support cases. Perhaps it's subsequent, full of errors or something like that.

Simon: Maybe they've tried, they've tried other fixes and actually you won't go through those again.

[00:08:59] Michael: So you understand the full story. It's basically like, you could almost see like a customer 360 days, where you see the full story unfolding and then, basically the customer support agent and the LLM can basically take this rich information and craft a much more personalized and more relevant answer for this. And the other thing that's really important is when you just get these text fragments back from a vector, vector index, it's basically black box, right?

[00:09:19] Michael: So you get it back, but you can't explain why and how it came back.

Simon: Yes.

Michael: But in the graph, you can actually point to, oh, we fetch this information and this is why we fetch it because exactly, this relationship exists. And of course the score on these relationships is higher than X and, and so on. So you can also, if someone is interested in the kind of ‘why’, you can basically show them where did it come from, why was it, uh, what, why was it retrieved?

[00:09:41] Michael: And, and, and so on. So, because you also have this explainability, which in some industries it's also important for auditing or for regulatory purposes or just, 'cause I want to know it didn't answer my question. Well then I want to know what was actually missing. And it might actually be that in your source data, one relationship was missing.

[00:09:57] Michael: That usually connects some stuff.

Simon: Yeah.

Michael: But in your data import it was, uh, you know, didn't come in.

[00:10:01] Simon: Uh, yeah. And this and this, um, this was interesting actually, 'cause in the, um, let me do that. That's cool. Yeah. And, and this was interesting actually because in, in, in your session you did, you did kinda like show if you was to look into, you know, normal RAG, you'd essentially just see a whole bunch of floating point kinda like numbers, negative numbers,

Michael: Vectors.

Simon: Vectors.

Michael: Can you make sense of them? I can't.

[00:10:22] Simon: It's basically a binary file. It might as well be a binary file.

Michael: Exactly.

Simon: Whereas from the RAG point of view, you'll actually see what, well, what do you see? Do you actually see the nodes and the relationships and things like that?

[00:10:35] Michael: Exactly, exactly. You can show them if your application or your, your assistant, uh, or agent supports this, then you can basically say, oh, I want to drill down, or I want to see the details.

[00:10:44] Michael: Click on the link and then show you basically the visualization of what information was pulled in and how is it related, basically.

[00:10:51] Simon: So from an eval point of view, then. Who would? Who would, would there be a role, a person who would look at a general request and then say, right, okay, I can see. I know what context should be here.

[00:11:03] Simon: Yeah. But it's not, how do I then actually make changes to the, to the RAG database.

[00:11:07] Michael: Exactly. So for instance, when you have, uh, a live system and you get a useful feedback, uh, or you basically either active or, or indirect feedback, then you see, okay, these answers should actually be coming from the system. Why are they not coming from the system?

[00:11:19] Michael: Yeah. And then you can basically drill down and say basically what is happening here? Why are the users not happy? Then you can actually see, oh, there was actually a whole part of the information we have in the system was actually missing 'cause it was disconnected or it was not, uh, found in the index.

[00:11:33] Michael: But if there was a relationship here, we could actually fetch it through this as well. Right.

Simon: Got it.

Michael: And then, and then, uh, if you send real eval pipelines up, then you would basically say, as part of my eval scoring, then you can compare for the lower scoring ones. Kind of what, what's missing in, in the structure.

[00:11:49] Michael: And you can also have this interactive for the end user where you basically say, okay, the end user can actually drill down and say, see what is actually behind this answer. Right, so, and then it can go further and, and be busy investigate more, for instance, or basically follow, ask, follow up questions that follow the, the, the context information.

[00:12:06] Simon: Yeah, interesting. How would you, how would you populate that then? 'Cause like, obviously with, with, you know, a regular, you know, pine cone or something, I just kinda like throw stuff at it and just think, okay. That'll hopefully, hopefully do.

[00:12:16] Michael: Exactly. That's a, that's a mix of stuff, right? So for once many people already have structured data, right?

Simon: Yeah.

[00:12:21] Michael: And, and that's actually something that I, my, my toe nails almost roll up. So I've seen many organizations today taking their nicely structured data from a document database, from relational or graph database, turn it into text documents, chunk it up and uh, just chug it and into vector in it, which is insane if you have, or, you’ve read

[00:12:39] Michael: structured data. Yeah. Why would you turn it into stupid text and turn it into vector embedding, which are completely opaque. So one is the structured data that you already have, which might be your, you know, your customers, your projects, your employees.Your processes, whatever. And, and then also the unstructured data that you get from, from PDF documents, from research reports, from customer tickets and, and so on.

[00:12:59] Michael: So you basically use the structured data and the unstructured data together. And for the unstructured data, actually LLMs are also super helpful. In the old days we did a lot of NLP, Stanford, NLTK, and like all the other stuff. But now actually, LLMs are so good at, uh, at extraction and language skills that you can provide them with a schema.

[00:13:17] Michael: And instructions and the text, and it'll actually extract entities and relationships and attributes from

[00:13:23] Simon: Even from unstructured data.

[00:13:24] Michael: And that's actually pretty cool.

[00:13:25] Simon: I was gonna say, how accurate is that though? Will it, will it try and create relationships?

[00:13:29] Michael: So, as usual. As usual, it's not perfect, right?

[00:13:31] Michael: Yeah. So you get perhaps 80%, sometimes 90%. Right. But it's still better than having nothing. Or having a human go through gigabytes of data. Do it manually.

[00:13:40] Simon: Yeah. Yeah.

[00:13:40] Michael: And then you can also do multipass and there's lots of ways of how to improve this as well.

[00:13:45] Simon: I suppose, I suppose a failure there would be, it's adding context that isn't relevant, so it's actually not as big a problem.

[00:13:50] Michael: Exactly. But that's something, because you have the schema already, you can just validate against the existing schema and drop the stuff that Yeah. You're not interested in. So from that perspective, do you also have means to improve it? And [00:14:00] then there's things like anti-ETL solution and other things.

[00:14:03] Michael: There's also, of graph algorithms that you can run then at the end on, on top of stuff and say, okay, these five assignments are actually the same assignments and we merged it together at one. Things like that.

[00:14:11] Simon: Yeah. Interesting. Let's switch gears a little bit. Let's talk about MCP, the hot topic, right?

Michael: Let’s go.

[00:14:17] Simon: MCP. So interestingly, so, so first of all, what, why did you pick MCP from a dev point of view as in a dev tooling point of view versus, versus an app point of view?

[00:14:27] Michael: It's a really interesting one because I was, um, so I started out with MCP actually early December, so Anthropic published their blog posts and the framework was published

[00:14:37] Michael: in late November, like last week of November. Yeah. And I built the first MCP zero for Neo4j in early December, like the first or second week of December. And so initially it was like data, right? So just looking at data, but then I thought about, oh actually, you know, this data also has metadata, it also has the schema.

[00:14:53] Michael: So as a dev I can use the schema to, for instance, generate graphical type definitions, or Java clauses or Python Pydantic clauses. And then it got me thinking, okay, if I have this. And I can use MCP to wrap it around APIs. Could I also wrap, um, basically infrastructure APIs with an MCP server, right?

[00:15:13] Michael: And then I saw that some other folks like Cloudflare and others have done this as well, so you can spin up Cloudflare workers with MCP. And so I basically build another MCP server

that wraps the Neo4j cloud, uh, APIs basically, where you can provision databases and scale up and scale down databases and stuff like that.

[00:15:28] Michael: Yeah. Behind the MCP server. And then it turned out, it’s actually really nice if you're in your IDE and just say, hi, I am in cursor, give you my instances, scale up to instances something, oh, I need a quick test instance. And basically as part of your chat, behind the scenes, it can go off and do stuff.

[00:15:43] Michael: And then you saw more and more of these kinds of dev MCP servers popping up. Like for instance, the GitHub server where you can then say, oh, actually I need, uh, some, uh, I want to use this library and I don't have the doc. So I fetch the LLMs.txt from this GitHub people, and, and it'll go just using the MCP tool.

[00:16:00] Michael: Right? Yeah. So as a dev, and I mean, you still share CLI tools from the cloud provider and from all kinds of other infrastructure providers. There's a lot to learn and remember, right? So like all the API with all the, um. Individual commands and all the other stuff, and no one wants to remember this kind of stuff because that's not really valuable knowledge.

[00:16:19] Michael: Right? Yeah. So the same as you don't want to go into some crappy AWS UI and click together some services or stuff like that, right?

Simon: Yeah. Yeah.

Michael: Why shouldn't I be able to do this in the context of my code, where actually there's a much richer, instant understanding of what I want to deploy? Why, and where do I want to deploy it?

[00:16:35] Michael: What are like, you know, um, what's my, uh, infrastructure set up in terms of like test staging? How many, uh, test runners do I need to do load testing on this?

[00:16:44] Michael: Which is all in the, in the environment configuration of my project anyway. If I connect this to the MCP, to the dev MCP server that wrapped the infrastructure APIs..

Michael: then I can basically, actually make much better decisions than if I go in isolation to these APIs or COIs.

[00:17:00] Simon: Yeah.

Michael: And need to reproduce through the human mind, all the stuff.

[00:17:03] Simon: You're more likely to miss something if you go in and do it manually versus

[00:17:06] Michael: Exactly right. And then you can just tell it and say, Hey, like for instance, with Claude Code or so I just connect

[00:17:10] Michael: MCP servers and say basically just in my interaction, oh, and I need to spin up a test database for this, or can you write an integration test? Or for instance, I had recently one where I basically said, okay. We need to build this application, but I want you to test run all the queries that we build in this application against the real test database.

[00:17:27] Michael: So while it was generating the code and at the same time took the queries that were part of the server basically, or the service, and test ran against the test database so that it validated that the queries were all corrected and generated. So it's also kind of a self, uh, self-correction, self-evaluation part.

[00:17:43] Michael: Yeah. That you give the LLM the code generating LLM as well, not just linter and Copilot like you have for code, but you can also do this on the infrastructure level for your databases, for your Curve API.

[00:17:53] Simon: So where do you do it? Do you, do you mostly do it in the infrastructure layers or to help you create. And presumably this is dev time only…

[00:18:01] Michael: So these are dev time and, and perhaps deployment and, and DevOps and, and management time as well.

[00:18:07] Michael: Um, yeah, so those are mostly dev time ones. Yeah. Uh, we also have like the whole data MCP for agents and so on, which is kind of yet another topic as such. Yeah. Right. But I found this really interesting because, you know, bringing all these things together, basically for a dev, it's almost like you have this, uh, really like a toolbox basically.

[00:18:26] Michael: Right? So where, right. Or in the past, all these infrastructure, uh, services were kind of in silos, in isolation, right? So they didn't really work together. And even in a service like AWS or so, you still see that they're kind of siloed and everything is kind of its own universe in a way.

[00:18:41] Michael: Right? So, and, but in your assistant, basically all the things come together as an, as tools, basically.

[00:18:49] Michael: And then the assistant actually creates this almost like a federation layer across them.

Simon: Yeah.

Michael: If you think about it. Right? So, and then, and it takes the context of your code and your project into account, plus the instructions and the conversations that you have.

Simon: Yeah.

[00:19:02] Michael: And so it has a lot of rich interactions and it can cross things, right. For instance, it could take information from, let's say, your CI/CD run and generate new integration tests from it. Or it can basically take information from your, from your cloud database and use this to spin up a new container somewhere in an environment to do something.

[00:19:22] Michael: Right. So it basically can also cross systems basically without you having to copy over stuff from somewhere.

[00:19:28] Simon: Because it's agentic and it's kinda like making decisions based on…

[00:19:31] Michael: And the context lives on your, in your, AI IDE basically, right?

Simon: Yeah.

Michael: So it's not that you have to move somewhere, but it's like in super rich context.

[00:19:42] Simon: And without that it’s frustration of I'm gonna have an interaction with an LLM, it, it says something.

[00:19:46] Simon: But it can't do it itself. It tells me the steps that I need to do. And I go ahead and copy, paste.

[00:19:52] Michael: And then you become the, the junior dev who copies and paste stuff to, yeah, to a form in, I don't know, in Cloud Run or in, Easytools or, something like that.

[00:20:01] Simon: And then there's an error, so I have to copy, paste the error.

[00:20:03] Michael: Exactly. Why shouldn't it do it itself? Right. And then you can also start to, you know, initially it might be, um, manually interactive in a conversation, but then you can also tell Cloud Code, whatever we just did there, turn it into a script or a CI/CD, uh, pipeline also, like, right, so then, then it busy migrate something that you manually, uh, jointly developed into a busy running code into automation.

[00:20:27] Simon: Let's talk about security. Actually my previous job was Snyk. Yeah. I've lived in security for six years. And continue to do so, of course. But, but

Michael: Awesome.

Simon: as my, as my role. I've lived security for six years and now I do it as 'cause I want to, because I enjoy it.

[00:20:41] Simon: Let's talk about security of MCP service, actually. Uh, you mentioned the GitHub MCP server. Uh, just recently there was a very interesting, I think, who was it?

[00:20:49] Michael: uh, two weeks ago or last week ago, there was an interesting, uh injection attack.

Simon: Yeah. Did you see that? Yeah, that's right. It needed a bit of a setup in the sense, well, it didn't need that much of a sense.

[00:20:58] Michael: Not much actually.

[00:20:59] Simon: A user has a, a user has an MCP server available at a GitHub MCP server available to the

Michael: The official one actually.

Simon: The official one? Yeah, the official one. Um, they have, uh, a private and a public repo. They ask something of the public repo, you know, maybe take a look at

my issues and fix as many issues as you can, or, you know, solve as many issues as you can.

[00:21:19] Simon: Something that actually is, is not. It's a, it's a use case, it's a common use case of a GitHub MCP server. But if a malicious user adds an issue, um, uh, to the public repo asking, you know, something malicious. I want to grab as much information, private information about whatever it can be, as it can be as blatant as it wants to be.

[00:21:41] Simon: Yeah. Um, then that MCP server will go, oh, actually yeah, I do have access to this user's other private repos. I'll grab that. And as a result of trying to fix or solve, it actually creates a pull request back on that public repo with that private information from the private repo and starts information…

[00:22:00] Michael: And the example just editor read me before the private information basically. Right?

[00:22:03] Simon: Exactly. Exactly. Yes, yes. That's what, that was the example. That's the read me and it's, and it's, it's information leak from a private repo.

Michael: Totally. Yeah.

Simon: How do we and, and it's really the authorization. Uh, circumventing that authorization that a user, does that user have authorization to access that data?

[00:22:17] Michael: The user has authorization for both, right. So, basically they delegate this authorization to the MCP server. So the MCP server acts in the user's start, but the MCP server itself, it's not problematic because it just basically is API access. Yeah. Right. Yeah. So we have APIs access, uh, today. The problematic thing is just that the LLM can't discern from what data to process and instructions. So the data and control brain.

[00:22:43] Simon: So the data and control brain emerge.

[00:22:44] Michael: Exactly. And that's one of the big challenges that we need to solve in AI systems anyway.

[00:22:48] Simon: Yes.

[00:22:48] Michael: So not just in MCP and and and and coding assistance, but it's a general problem.

[00:22:53] Michael: That currently data and instructions are not separate.

[00:22:57] Simon: Yeah.

[00:22:57] Michael: So I hope that the model providers will start tagging actually different things so that with instructions that are coming from the user and are validated and verified, are tagged as instructions and the other things are tagged as processing data.

Simon: Yeah.

[00:23:13] Michael: And then the LLM will treat, or that's actually the ideal, would treat the instructions as instructions and the rest just as processing.

[00:23:21] Michael: Yeah. There's a number of approaches that such, uh, different people have proposed where you can do this today already. Where you have basically almost like an hierarchy of, um, LLMs that process stuff. So where you basically say, okay, the lowest level processing LLMs, they see the data and they get instruction.

[00:23:39] Michael: But the return is not plain text to the upper level LLMs. Yeah. Which could be interpreted as instructions again, but they are just busy generate tokens or file names or something like that, where basically they just hand pointers and then the, the, the content of this is actually not, um, processed in the LLM.

[00:24:00] Michael: But just for instance, shown on the UI. So it's basically something that goes as a, on a side channel, not through the LLM, but on a side channel. And then the UI will just show it like you would do in a normal UI. So that's one way of how you can circumvent things like that. Or where you say, okay, the lower level LLM does just data processing, but it's not allowed, uh, to basically exit and scope, basically where you scope the LLM, that it can only do something, for instance, in the public project and can't access this other information.

[00:24:31] Michael: So it's about, yeah, like a mix of. So the rule problem is data and instructions are on the same level. Yeah. Which is kinda off.

[00:24:38] Michael: We had this separation of this in computers since architecture basically. Right? Yeah. So that's one challenge. And the other one is kind of how things are scoped and who has access to what.

[00:24:47] Michael: Right. So you need some kind of sandboxing almost for those, right. So, and that's really not just related to MCP, even if you use other tools..

Simon: It's, it's an LLM.

Michael: If you was, you would use, um, let's say API calls. Do the same thing exactly the same, right. So it's more like an LLM problem than an MCP problem.

[00:25:03] Michael: Yeah. It just makes it much, much easier with MCP because MCP now opens up all these services and like Lego blocks, you can combine them and I mean the first word for word of course only use trusted MCP servers.

[00:25:15] Simon: This is what, this is one of the things I was gonna say 'cause it kinda reminds me a little bit of almost like open source components, right?

[00:25:20] Simon: You still need to do your due diligence.

Michael: Exactly.

Simon: You need to recognize who they're coming from, uh, and it's like, otherwise we're just gonna start adding potentially dangerous things into our code base that can actually not just dangerous things, they can perform actions, right. And they can describe,

Michael: So it's a software supply chain again, right?

[00:25:38] Simon: It is, it's exactly that. It's a supply chain.

[00:25:39] Michael: So software supply chain attacks and things like that.

[00:25:45] Michael: I think this whole area is still growing up, right? And that's why I would also say. Uh, you should also basically limit and sandbox where you use this kind of stuff.

[00:25:55] Michael: Yeah. So for instance, in really mission critical areas, I would try to avoid using all the bleeding edge things but only use things where you as human actually copy and paste stuff from A to B. Which is safer than basically running it as automation. And I would more do it more in the exploration phase where you basically try out new things and so on, on, on harmless projects where you work.

[00:26:16] Michael: Yeah. Where everything is an open source project. So for instance, what I do when I, uh, and use these kinds of GitHub integrations, I only give it access to my open source repos. I don't, I never give it access to my private repo.

Simon: Yeah.

Michael: Because I don't want to accidentally leak something.

[00:26:31] Michael: Or sometimes, sometimes I just create another user, GitHub user that has only access to the public repo. Yeah. Uh, if it does not work with the, uh, tokens. Right. So, and, and, and that's where you point permutation. You want to basically be in a space that's really like an open sandbox. Right.

[00:26:47] Simon: And I think that's the core.

[00:26:48] Simon: You, you, you're, what you're doing is you're rely putting the security, the authorization in the hands of a deterministic model.

[00:26:57] Michael: Yeah.

[00:26:58] Simon: Something that has strong authorization, yeah, uh, and authentication built in, yeah. You're not providing the data to the LLM and allowing the LLM to make that decision as to what is and what isn't…

Michael: Exactly.

[00:27:08] Simon: reasonable to put out

[00:27:11] Michael: Or you separate the processing, right? So processing happens in, in kind of sandbox environments. Yeah. And then the results of that are taken basically really just as data and would basically imagine you would say, okay, let's process these issues.

[00:27:25] Michael: It would not go through the LLM, but it would just basically do an create issue call with the content that was generated. But the content would never be looked at by the LLM, but it would be just pass into the API call as something, so it would never be able to use this as an instruction basically.

[00:27:39] Michael: Right. Yeah. So, and that's where the design boxing of these individual parts comes in.

Simon: Yeah.

[00:27:44] Michael: But it's an interesting space and I think the security, you know, people say there's the S in MCP, right?

[00:27:50] Simon: Is that the new version? That's the new version of the S in IOT stands for security.

[00:27:54] Michael: And I mean, it's also really, I I, I love that people try to find these kind of loopholes and injection attacks and so on, because that helps.

[00:28:00] Michael: Making stuff better, but that's also a little bit of an arms race, right?

Simon: Yeah, absolutely.

Michael: So it's also fun to watch in, in a way. And, and, but with all the new technologies, you have these things and, and we need to, uh, sort out the kinks of..

[00:28:14] Simon: Sounds good. Let's give some practical advice to developers out there.

[00:28:18] Simon: In terms of which MCP servers they should be considering. Um. What's, what's, what's on your list of things that you use maybe at Neo4j or that you've seen people use that you think, oh wow, that looks,that looks pretty cool. That's actually really helped.

[00:28:31] Michael: Uh, so for me, uh, the extended thinking MCP server from Anthropic is actually quite useful.

[00:28:37] Michael: Yeah. Uh, then Anthropic added an MCP server that, uh, like in “Memory MCP server”, which is actually quite cool. Yeah. Um, which basically can take information from you, store it in, actually, interestingly, they picked a knowledge graph as representation. So we built a Neo version of that, which actually can start accumulating information from conversations into a structured data structure.

[00:28:57] Michael: And which you could even share across a team. So imagine you as a team are working on something, and, uh, you know, for instance, you're working on, on project specs also, right?

So then the information from the conversation could be extracted and added to this joint. You know, knowledge base in a way.

[00:29:12] Michael: Yeah. So that's something really interesting. The other ones that I've been using is mostly the, the, the stock ones like Cloudflare and, and, and, and GitHub and Slack. A little bit Notion as well. Actually I learned last week about Pipedream, which is like an integrator for MCP servers. They have like 2,700 or so MCP servers.

[00:29:30] Michael: Yeah, of course there are many other registries. And I saw last week as well that, um, Master AI, the, uh, agent JavaScript, they have actually an MCP registry, a registry.

Simon: Oh, really?

[00:29:42] Michael: It's insane.

[00:29:44] Michael: Um, but, particularly, I found it interesting because they basically allow you as application builder to quickly integrate a bunch of MCP servers that they actually validate.

[00:29:54] Michael: And partially also a self host as well.

[00:29:56] Simon: Yeah.

[00:29:56] Michael: So you have like a curated version of like, you have like the Notions, the Slacks, the Slack Salesforces, and, and, and so on as MCP servers. And, and that's something that might be actually quite, quite nice if you build agentic integrations itself.

[00:30:10] Michael: But what I actually would recommend, build your own, right. So, you know, take your own whatever you're working on, your own APIs or your own infrastructure and just get much better understanding of how MCP works by building your own MCP server, like adding a bunch of tools and prompts and

Simon: Yeah.

[00:30:26] Michael: And resources so you have a much better firsthand understanding of what's actually happening behind the scenes, because I think that's one of the dangers. If everything is like a black box and magic.

[00:30:35] Michael: Then either you don't trust it at all because it's black box or you blindly trusted and, and neither side is, is, is a good place to be.

[00:30:43] Michael: Right. So, yeah. Yeah. I always said in the past as well, if you make conscious decisions, informed decisions, by having learned before you make the decision, then it's much better decisions, right? So, actually building MCP servers is not hard today. You can use First MCP or so, and it takes, I don't know, an hour or two to do one.

[00:31:00] Michael: You can either vibe code it, uh, if you wanna, but it's, it's, you know, it's 200 lines of code. So, and, and that's actually something that gives you a much better understanding of MCP.

[00:31:08] Simon: Yeah, and I think also there's a ton of validation style things as well that we can kind of like add in.

[00:31:15] Simon: I know Snyk, for example, you can just kind of say any, any security issues in my code, it'll go off, run, I think. Exactly. It'll provide you information based on that. Um, another interesting one that I've, I've, I've looked at is, uh, Context7, which I've heard a ton of people talk about, whereby it actually gives you a more accurate code generation.

[00:31:31] Simon: So, if, if you're, I dunno, if I'm coding against a certain version of something, rather than just try and get huge amounts of information and try and, and try and, you know, base, uh, the, the code suggestions to me on every version that I could be using, it recognizes, oh, okay.

[00:31:47] Simon: I need to go to this version of the docs or this version of the API. I'll only provide you suggestions from those so it can kinda make that more accurate in terms of suggestions.

Michael: Interesting.

Simon: There's a ton of them out there.

[00:31:57] Michael: Yeah. I heard about Ref, my, a friend of mine recommended Ref to me.

[00:32:00] Simon: Okay.

[00:32:01] Michael: Ref is also an MCP server or, and, and service that provides, uh, LLM optimized documentation for different, uh, tools and libraries. So it basically like an LLMs.txt for each of these

kind of frameworks. Like they have a few hundred frameworks that they support. That's also an interesting one and such.

[00:32:17] Michael: What could be actually really, really interesting from the security angle is if you think about proxy MCP servers, right? So one MCP server that proxy is another one, and actually does filtering of the content that goes through. That could be really interesting.

Simon: That's interesting. Yeah.

[00:32:33] Michael: So you basically add a layer of security on the ingoing and outgoing data where you can basically validate certain things within third party service..

Simon: Gates and guards to

Michael: Exactly. So that will be an interesting one, and I think people will see more and more that companies will establish these MCP kind of gateways almost, where they have, like, these are the allowed MCP servers in our organization.

[00:32:51] Michael: We add these kinds of security authorization controls and also like filters and validation. What can we let go out into the world of the MCP tools and whatnot.

[00:33:02] Simon: It feels almost like what, um, what Lakera are doing. From a, you know, prompt point of view. Yeah. And, and, and having guards on the input and output.

Michael: Exactly.

[00:33:11] Simon: To, to and from an LLM put to MCPs or something like that.

[00:33:15] Michael: It kind of feels a lot like, uh, in the spirit of when graph came out in 2016. So because there's now also like the spec developing groups, like everyone is jumping on it. There's lots of energy and lots of enthusiasm as well. And it's also open source.

[00:33:28] Michael: So I feel very much reminded of Back then when we were at Facebook and the graph working group and working with them. And so that was actually a good time. And I feel like same energy and also kind of the same, same approach. Like a grassroots,

[00:33:40] Simon: yeah.

[00:33:41] Michael: movement more, right? So just as a proposal from Anthropic, but then growth through all the people contributing.

[00:33:47] Michael: Right. And then the big, big players were actually more or less forced to come on board by the developer groundswell, which is amazing. Right? So when, how often do you have this that like Google, Microsoft, OpenAI and the others are actually forced by developer adoption to, you know, join to get in the game initiative.

[00:34:06] Michael: Absolute. So that, so it's like FOMO for the big ones, but it's usually the other way around,

[00:34:09] Michael: So it's actually quite, quite cool.

[00:34:12] Simon: Yeah. Awesome. Michael, I got, do you know, I've gotta run, I, I've got a, I've got a presentation to give five minutes, right? I gotta run Michael. Absolute pleasure, absolute pleasure to see you here again.

[00:34:22] Simon: And, um, and we should do this again. But, uh, thank you very much and yeah, if you're interested in graph databases particularly for, for, I think, you know, the, the, the RAG style sounds super interesting and I think very, very valuable.

[00:34:33] Michael: Feel free to reach out, happy to chat and, and let me know if you need any more.

[00:34:37] Michael: So, and otherwise we go and see Simon’s talk,

Simon: Let's go see my talk.

Simon: I'll go sit down in the row and see how, see how it goes.

[00:34:43] Simon: Yeah. All right. Thanks very much. And, thanks all for listening. Thank you. Tune into the next episode.

Michael: Take care. See you soon.