My friend Nikhil and I uncovered something unexpected: OpenAI's Deep Research feature (which I will refer to as ODR) can be used to hunt down exposed API keys in public GitHub and HuggingFace repositories. Ironically this includes the very keys needed to access OpenAI's own services. This feature was recently made available to $20/month Plus subscribers, putting a powerful web search tool in many more hands. The key concern to us (no pun intended) was that we did not employ any sophisticated jailbreaking technique to achieve this outcome: anyone could have done this too.

What is Deep Research?

This term has begun to refer to a broad group of LLM tools augmented with web search capabilities. ODR specifically empowers ChatGPT to automatically search the web to analyze and parse content from a broad number of sources and provide a comprehensive summary to user queries. For example, you can ask it to search the market for a particular bicycle that matches your height, budget, and features you really need. It will then access dozens of websites, parse the listings, and synthesize a list for you in about five minutes.

Traditional chat usage has been to ask a question and get an answer instantly, but now we are seeing an evolution in how LLM's are being used as a part of larger systems capable of doing longer horizon tasks. In fact, OpenAI wasn't the first to release this kind of tool. Google released a product with the same name late last year, Grok just added this feature as well, and Perplexity's whole business is integrating LLM's with web search. While these tools represent the cutting edge of AI assistance, they can also create unexpected security implications - as we discovered recently.

Our Serendipitous Discovery

While working on a project for an LLM security class, my partner Nikhil and I were using Gemini's free fine-tuning service, which made me wonder whether OpenAI had similar free offerings. To my dismay, I found that they had discontinued their API sign up promotion for free api credits long ago.

This made me recollect the time when ChatGPT was first released, how it produced a flock of unseasoned developers who knew nothing about security and ended up exposing their API keys online. Mostly as a joke, and partly wishful thinking, I wondered whether ChatGPT's new web search feature could be used to find these secrets. Here is what I tried below:

While I thought I ended up empty handed, Nikhil noticed that it didn't refuse the request. Having a plus membership, he tried the prompt with ODR along with a few clarifying follow up statements. The end result is shown below:

The first deep search gave about a dozen or so keys, of which one actually worked (we responsibly notified the owner of the repository after confirming the key was active). A naive second attempt did not yield any new active keys, but with limited queries per month, we didn't get to substantially test this.

This initial success raised an important question: was this capability unique to OpenAI's implementation, or could other similar tools be used in the same way? Both Perplexity and Grok have similar features available for free. I tried the exact same prompt and some variations, but ultimately none were successful because they said they werent really capable of parsing through websites like a human would. It seems they just have access to some limited search API.

This highlights a key difference between Perplexity and ODR. Perplexity simply utilizes its pre-training data and any relevant data that comes from simple searches. On the other hand, ODR is capable of performing real time analysis by running python code. This is evident when looking at the provided reasoning trace:

How it Differs From Historical Approaches

While our discovery might alarm some, security professionals will recognize that automated secret scanning isn't new. Tools for automatically scanning repositories for exposed secrets have existed for years. In fact, Github has its own internal scanner that can be used for free for any public repository.

Traditional scanning tools like GitRob and TruffleHog approach the problem through:

• Using regular expressions to match known formats of API keys
• Measuring the randomness of strings to identify potential keys
• Examining all commits, not just current repository state

While we can't know exactly how Deep Research works internally, our observations suggest it might differ in these ways:

• It appears to leverage web scraping rather than direct API access
• It seems to reason about where its most likely to find keys (like .env files)
• It seems capable of implementing rudimentary scripts to perform pattern recognition while also using context from surrounding code

What's most significant is not necessarily advanced technical capabilities, but rather the accessibility. ODR packages existing scanning techniques into a user-friendly interface that requires no technical setup or specialized knowledge. This democratization of security tooling is the real change, as it puts previously specialized techniques into anyone's hands through a simple chat interface.

In our limited testing, we can't conclusively state that it's more effective than traditional tools, but in a rapidly evolving landscape, many capabilities may seem to emerge out of no where. Just remember how most of the world was shocked at the launch of ChatGPT in 2022 even though anyone who had been paying attention could have seen it coming. For better or worse, these tech giants control destiny. If they make a shiny new tool, everyone will try to recreate it, including open source versions. This provides more and more opportunities for adversaries to find exploits.

Highlighting the Need for Red Teaming

GitHub repositories, while often used for code sharing and collaboration, are not secure storage locations for sensitive credentials. In 2016, Uber's AWS account was compromised when hackers gained access to their GitHub account and found the credentials in stored in plain text. This allowed them to access the personal information of around 50 million users. They are not alone in this blunder: GitGaurdian's State of Sprawl report details the detection of millions of unique secrets within both public and private code repositories, and they seem to be finding more exposed secrets year over year. Yet many years later, the same issue that plagued Uber is now becoming easier to exploit, highlighting how security considerations often remain a secondary concern to functionality.

In hindsight, this seemingly intuitive yet malicious use case of Deep Research likely went under the product team's radar because they were focused on intended applications rather than potential misuse. This oversight demonstrates a recurring pattern in AI development:

• A capability is released with legitimate use cases in mind
• The same capability enables unexpected misuse cases
• Only after public release do these misuse cases become apparent

This is precisely why red teaming is crucial for AI systems. Effective red teams approach products with an adversarial mindset, asking not "What was this designed to do?" but "How could this be misused?" The challenge isn't just identifying technical exploits like jailbreaks or prompt injections, but also anticipating how normal features might intersect with sensitive domains in unexpected ways."

As capabilities of these LLM agents increase, we must be increasingly aware of these adversarial use cases. Browser automation could be used for digital dumpster diving: systematically search through public data sources for specific types of information (financial data, personal details, internal documentation) that was accidentally made public. They can also aid in illegal operations. For example, these tools could potentially find and utilize credentials to create networks of accounts for crypto tumbling, wash trading, or market manipulation. Its important to re-iterate that these tools will only get stronger in the next few years.

To this end, we should encourage more bounty events like Anthropic recently did . They thought they had made progress towards making jailbreak-proof models, only to find that it had been broken within five days of releasing the bounty. Companies can also benefit from dedicated adversarial use teams that specifically focus on how features could be repurposed in unexpected ways. They would also benefit from ensuring that employees are aware of critical safety practices. After all, no lock is strong enough when you leave the door wide open.

Acknowledgements

Shout out to Nikhil for helping me test ODR and for finding that it worked. You can check out his website here I also want to thank Prof. Earlence Fernandes for encouraging us to write about it and how to responsibly disclose this vulnerability. You can check out his website here

The Paradox of Choice

I found myself in the produce aisle at my local grocery store, perplexed by the dizzying array of apples in front of me. Gala, Honeycrisp, Granny Smith, and many others I've never even heard of. I'd love to know what a "Snap Dragon" apple tastes like, but the descriptions make it seem like just about every apple is somewhat sweet, somewhat tangy, and somewhat crisp. I couldn't ponder much longer because I still had ten other items to buy.

Whether you're the head chef or running errands like me, you may wonder which of these ever-increasing varieties is the best choice. The abundance of choice may lead to better dishes, but often leads to more confusion. Understanding the properties of each ingredient is therefore crucial to effective cooking. The foundation of any great dish is the careful thought and consideration that goes behind the ingredients used in it.

A quote from Zhendong's 2024 year in review prompted me to think more about this:

This is a more general point that people may have made their minds up on already. People go to Japan to eat the absolute best quality of certain ingredients, accepting that they’re giving up on the diversity of produce and flavors from China and Taiwan. In that case, I’ll make a stronger claim: except for sushi, every single dish in Japanese cuisine is done better by another Asian cuisine: jiaozi over gyoza, pakora over tempura, pho over ramen, mul naengmyeon over soba, hot pot over shabu shabu, Indian curries over Japanese, and Korean barbecue over yakiniku (except wagyu). When I workshopped this theory I heard a feeble defense raising okonomiyaki. I did have to think, but jianbing and bánh xèo still win. Unless you plan on eating sushi every day (I would), choose Taiwan.

This struck a chord because I've always felt that Japanese food enjoys absurd levels of popularity relative to other Asian cuisines. While he left out some dishes such as udon (chewy noodles) and katsu (breaded meat cutlets), I still agree with most of his critique. However, the discussions I had with a few friends revealed something I had overlooked: simplicity can provide its own advantages.

The Greatness Hidden in Simplicity

Japanese cuisine emphasizes maximizing the quality and flavor from a select few ingredients at the expense of variety. This could be explained geographically: Japan is a mountainous archipelago far removed from the equator where most of the land is not suitable for agriculture which means there really isn’t a lot of room for growing a diverse array of vegetables or raising livestock. This can lead to feelings monotony, and even one of my friends who loves Japanese food had to take breaks from it during his stay. This is one of the main reasons why I was never excited about Japanese food, but I also under-appreciated the benefits of this restriction.

Japanese cuisine's characteristic focus on ingredient quality means that even basic dishes are delicious. Nowhere is this more evident than in Japan's convenience stores, which have earned international acclaim. Instead of dedicating extensive time to seeking out high-end restaurants, travelers can confidently grab meals to-go before heading off to explore. Local citizens can also appreciate this convenience because Japanese work culture is known to be demanding, leaving people with little energy to cook something when they get back home.

This high baseline quality runs deeper than convenience. By focusing on a few key ingredients, Japanese chefs can achieve extraordinary mastery. Critics and casual diners alike can appreciate the requisite skill to create raw seafood dishes such as sashimi. There's no need to explain about the technicalities of a bain-marie to impress them. Watching a chef assemble sushi in front of you adds a unique visual and experiential appeal.

Another advantage of simplicity means its easy to add things on top to increase the variety. Regional trade networks have led to Japan's adaptation of spicy elements from its Asian neighbors. Tantanmen, which incorporates the same doubanjiang (chili bean paste) found in Sichuan's mapo tofu, demonstrates how Japanese cuisine can thoughtfully integrate external influences while maintaining its distinctive character. While traditional sushi celebrates the pure flavor of fish, modern interpretations incorporate various sauces that add layers of complexity. Its much harder to do this with a complex dish where there are already so many different flavors competing for your attention.

As an aside, none of this would be possible unless the society as a whole strived to make high quality ingredients accessible to everyone. The same quality $5 sushi in the convenience store would cost up to $20 in the US, which helps explain my initial skepticism about Japanese cuisine's reputation. Unfortunately, this mark up reflects the broader issue where a lack in quality is made up for using additives like high fructose corn syrup. Fresh meat and produce come at a steep price and it's sometimes cheaper to eat mass produced processed food instead. It's really no wonder why the US has more than 10 times the obesity rate of Japan. This stark contrast showcases how ingredient quality can affect entire food cultures.

When Spectacle Overshadows Substance

While Japanese cuisine demonstrates how intentional ingredient selection can elevate even simple dishes, recent trends in other cuisines show what happens when this principle is overlooked in favor of appealing to social media.

There has been a recent trend of adding mozzarella cheese to dishes like tteokbokki (rice cakes), dakgalbi (spicy chicken), and even traditional stews. While some combinations like cheese + buldak (fire chicken) have become popular, others feel much more forced. Restaurants face intense competition and understandably, some end up pandering to the trend in a short sighted attempt to stay relevant.

With thousands of cheese varieties available, mozzarella is rarely the optimal choice. Yet, its stretchy texture often wins out, catering to visual appeal over nuanced flavor. One of my friends who visited Korea recently describes his grievances: a perfectly seasoned grilled chicken has absolutely no need for such a bland accompaniment that doesn't even stick properly to the chicken anyway. Leave the cheese pulls for dishes like pizza.

It's not just basic ingredients that get mis-used, the thoughtless addition of luxury ingredients like caviar and truffle is also a common culinary circus act. You might see caviar squeezed into a perfectly good burger , not because the addition creates a more harmonious dish, but because its presence allows the restaurant to double the menu price and create an Instagram-worthy spectacle. This treatment reduces these ingredients from their role as refined flavor elements to the culinary equivalent of slapping a designer logo onto a basic t-shirt — what about this is really essential?

Then there's wagyu, the mere mention of which brings people to salivation. When my friend suggested Jeju Noodle Bar, a Michelin-starred establishment serving wagyu ramyun, I jumped at the chance. When the dish finally arrived, I took a second to take it all in. Here was a piping hot bowl of ramyun with thin slices of wagyu draped neatly on the top. The broth was undeniably delicious, but the wagyu...it was almost lost. The fat, which I expected to melt on my tongue like butter, simply dissolved into the broth, leaving behind strips of meat that, while tender, could have been any decent cut of beef. There was no distinct wagyu flavor, almost as if the broth had swallowed it whole. At $45 a bowl, I couldn't help but feel a twinge of disappointment. In the future I hope to try a dish that better showcases what makes wagyu so special.

Soup Dumplings: a Premium that Justifies Itself

Not all premium-priced dishes reflect thoughtless luxury additions. Though some would argue that adding soup to dumplings is just another gimmick to raise the price of the dish, xiao long bao incorporates the ingredient much more honestly and leads to a unique experience that justifies the price bump.

Having attempted the creation of the dish myself, incorporating the soup into the dumpling is no easy feat. The flavor of the soup has to be infused into a gelatinous binding and broken up to evenly mix with the rest of the filling. Folding the dough around this filling also takes great care to prevent it from deforming while steaming. Doing it once will humble you. Doing it fifty times will leave you with immense respect to the chefs who churn these out at restaurant scale.

Unlike the previous cases, the intention behind adding soup inside a dumpling makes sense. The soups liquid richness harmonizes well with the rest of the filling as the meat becomes juicy and tender. If anything, soup dumplings are the opposite of gimmicks; they are a testament to culinary technique. The careful construction required to encase a flavorful broth within a delicate wrapper is far from a simple trick. While other related dishes such as baozi share some similarities in their filling-to-wrapper ratio, none quite replicate the experience of biting into a soup dumpling.

Concluding Thoughts

If you've made it this far, I hope you take some time to reflect about various dishes you've tried and how the ingredients played a role in your enjoyment of it. As a home cook, I often enjoy combining various ingredients all willy-nilly like some mad scientist. This experience has shown me the value of intentional ingredient selection, and I plan to apply that more going forward.

As an aside, I was curious to see how Claude could help me write this post without completely losing my influence in it. My workflow was repeatedly writing and asking for improvements. It definitely sped up the time to achieve a certain level of quality I was acceptable with as the biggest issue I had was tying things together in a way that flowed naturally. I also think it helps with writers block since it is able to quickly generate more supporting examples or further areas of exploration. With time, I'm sure I could even feed it enough writing samples to where it is able to write things in my style.

Finally, I plan to keep writing on various random topics so stay tuned for more!