My blog is 6 years old with not frequent posting. I was looking to get feedback of my content so to sense where I have to improve and push myself to dedicate more time for a better quality writing. I contacted a couple of bloggers who earned authority in the field of IT and cybersecurity, only one responded to me and it was a gentle excuse, a busy schedule as she said.

Hopefully AI is busy only when it is thinking. AI, the de facto tool that should augment wetware’s intelligence to a satisfactory outcome, can be deeply deceptive.

So, using GPT-2.5 Pro, I provided the following prompt to gauge the blog content in terms of quality and the level of seniority:

The model started the response with a generic summary that when it unfolded it came up with unrelated topics, and none of the my posts has been referenced in the response:

Surprisingly, the following reminder prompt didn’t help and the model responded with other unrelated topics like “The Ultimate Guide to Mastering Linux for DevOps”, “Demystifying Kubernetes: A Beginner’s Guide” :

The round-trips that follow hadn’t make the expected trajectory correction. But what surprised me the most is why the model hasn’t be able to make it clear he has no knowledge of the blog’s content, and decided to put “effort to appear credible” with fabricated topics.

Conclusion:

This is a write-up from an end-user perspective. Users shouldn’t need to remind the LLM not to fabricate data when it doesn’t have access to a specific dataset.

Obviously I have “consulted” with the other LLMs and the outcomes were just right. However, this is one of the examples where we have to warn ourselves that using LLM for a topic or a dataset that’s new to us must be done with absolute due diligence.