Build for AGI in 2025

As soon as I gained access to OpenAI o1 model by API, I ran the latest o1, gpt-4o and claude sonnet LLMs through my private CFA level 1 exams.

Now o1 has officially scored in the 99th percentile with a correct rate of 90.7%, without any access to CFA textbooks or mock exams, just a calculator and good prompts.

CFA Level 1 Chapter Total Questions Correct (gpt-4o) Correct (sonnet) Correct (o1)
Ethical and Professional Standards 126 74.60% 73.81% 82.54%
Quantitative Methods 91 75.82% 75.82% 94.51%
Economics 84 88.10% 90.48% 94.05%
Financial Reporting and Analysis 160 78.13% 78.75% 91.88%
Corporate Finance 67 79.10% 71.64% 89.55%
Equity Investments 85 74.12% 69.41% 94.12%
Derivatives 43 74.42% 83.72% 95.35%
Fixed Income 89 73.03% 71.91% 88.76%
Alternative Investments 37 83.78% 81.08% 97.30%
Portfolio Management 57 77.19% 75.44% 85.96%
Total 839 77.47% 76.76% 90.70%

Due to its proprietary nature, CFA materials are not part of a model's pre-training data set. These benchmarks are therefore out of sample.

It's worth noting that o1 scored particularly well in investment-related chapters which are the most relevant to the target users of SigTech.

I expect the o1 performance in "portfolio management" and "fixed-income" to be significantly improved if it's given access to our quant framework as tools.

I also expect the incoming o3 model performance to be >95% correct rate, i.e. 99.9th percentile.

I define AGI as a LLM that's able to do almost all knowledge work in the 99.9th percentile. I fully believe that it will be here by the end of 2025.

It will be expensive to start with but the cost is on an exponential curve of decreasing 90% every 6-12 months.

In 2025, our entire business including our product MAGIC and organisation itself must be built for the arrival of AGI. There is no playbook so we have to think creatively and from the first principles.

I hope everybody has a good holiday to rest and reflect. See you in 2025.

AI reviews EU AI Act

I used OpenAI o1 reasoning LLM to review the EU AI Act.

It took o1 only 18 seconds to think. It took me 2 minutes to convert the PDFs to nicely formatted text.

My prompt is “Give me the top 3 logical contradictions in the EU AI Act I've pasted here: TEXT

Here is the o1 response:

1. A "risk-based" approach that isn’t always risk-based:
The proposal emphasizes a proportionate, risk-based regulatory model, claiming that requirements scale with the potential harm of an AI system. However, the classification of high-risk AI systems is often triggered by the sector or intended use listed in Annex III—regardless of actual, demonstrated risk levels. This means certain AI applications are subject to stringent obligations purely because they fall into a predefined category, contradicting the idea of truly tailoring requirements to the real and context-specific risks posed by the technology.

2. Encouraging innovation while imposing costly compliance burdens:
The Act states it aims to foster innovation and support small and medium-sized enterprises by providing regulatory sandboxes and other guidance. At the same time, it imposes stringent conformity assessments, continuous monitoring, and record-keeping obligations for high-risk AI systems. These demands can become a heavy financial and administrative load, especially for smaller providers. This tension between claiming to nurture innovation and simultaneously mandating complex, resource-intensive compliance contradicts the stated objective of reducing barriers and encouraging market entrants.

3. Harmonizing the single market but layering multiple standards and bodies:

The proposal seeks to create a uniform framework to prevent market fragmentation, ensuring that AI systems can freely circulate throughout the EU. Yet, it references and depends on numerous existing and future sectoral regulations, the involvement of multiple notified bodies, and potential reliance on a patchwork of technical standards and common specifications. As a result, developers and users may navigate a complex web of overlapping rules and authorities. This complexity undercuts the promise of a clear and harmonized single market for AI, creating, in practice, a contradictory situation where the pursuit of uniformity may instead lead to confusion and uneven application.

I’d love  to run this prompt in o1 with all government documents.

Share a Laugh

One of the best things I did this year was to take a 10-week stand up course by comedian Logan Murray.

It culminated in a live performance before 100 people at the Museum of Comedy, which is located next to the British Museum in London. 

When I invited close friends and colleagues to the show, most were shocked. “That is so scary. Why would you do that?” they wondered. 

I didn’t do it for the reasons many do. I wasn’t trying to improve my public speaking skills or reduce stage fright. I certainly wasn’t preparing for a side hustle or career transition. 

It was more philosophical. Humour is so fundamental to human nature. I wanted to understand what makes a joke funny and why people laugh. How is humour related to curiosity and inquisitiveness? 

I knew my friends would understand once they saw the show. 

They are all curious, intelligent people. What I love most about them is that they share a sense of humour. They don’t take themselves too seriously. Over the years, we’ve bonded over laughter far more often than over food, drinks or holidays.

There is something about “sharing a laugh” that’s both primitive and profound. 

I took the stand up comedy course to learn about this eternal form of human bonding.

Over 10 weeks, I studied with a group of total strangers. There were three semi-pro comedians, one ex-talk show host, a former concert singer, one private equity analyst, an entrepreneur, a former professor, a body guard and me. 

We were taught key comedic concepts of status, spontaneity and storytelling. We critiqued each other’s performances. Everyone had a distinct style. We rarely talked about what we did in “real life.”

It felt strange to know so much about each other and yet nothing at all.

But it was a wonderful bonding experience and a beautiful memory.

Like the memory of visiting Greenland with total strangers in 2016.

Know the Game

Mathematical analysis was my nemesis.

I had nightmares of “infinitesimal quantities”.

One of the biggest shocks of my life 25 years ago set in motion the chain of events that led me to start a financial AI company, SigTech.

It starts with my roots growing up in Shanghai, China’s largest city with 25 million inhabitants.

It’s a vibrant place filled with entrepreneurial energy. In Beijing, when people go out, they gossip about politics. In Shanghai, people talk about money. 

When you grow up in China, you get used to the idea that whatever you do, a million other people are trying to do the same thing. You are constantly under pressure to compete because it’s drilled into your brain that no one's going to give you anything. You have to earn it.

You have to be an excellent student and go to excellent schools to get an excellent job. That’s the way it works. So, I worked hard to get good grades and won national Olympiads in math and physics. Being a top student was my identity.

When I was given the chance to choose any major in any university in China, I chose to join a class of 20 students of “special talent” assembled at Shanghai Jiaotong University, one of the world’s top universities.

I quickly realized I was not going to be the best. In the mathematical analysis class I took, I was ranked in the bottom half.

It was shocking!

Even worse, no matter how much effort I put in, I simply lacked intuition for “epsilon”.

At the time I thought it was inconceivable. It challenged my self-identity. If I can’t be the best in math, then who am I? What will I do with my life?

It was a critical period and it prompted me to start to think more about what I wanted, instead of what I was supposed to want. It’s not a common thing in China.

The reflection prompted me to take a different path than I had planned, eventually applying to Cambridge University for a doctorate, and working in finance in London afterwards.

Those lessons helped me, as a computer scientist, transition into broader roles, first at Barclays Capital and then at Brevan Howard.

I developed a sense of independence, fostered by the variety of jobs. When people told me to do something, I was less likely to blindly say yes. Instead, I asked more questions.

Looking back, the thing that probably contributed most to my professional development was being forced to sit in that room of 20 other students at university and acknowledge I wasn’t ever going to be at the top of the class.

Sometimes it takes realizing you aren’t as good at the game as you thought, to change your mind about the game you want to play.

Like a hunting cheetah in the Kruger National Park, South Africa, 2018

The Power of No

Everyone wants a customer to say YES.

I want a customer to say NO.

A salesperson wants affirmation from a customer. He goes into a meeting to pitch: look at our amazing product with amazing features; how it can help your use cases; how it can change your life.

The customer smiles: “Very interesting! Definitely something I’d consider. Keep me updated.”

The salesperson is pleased, thinking that the customer loves it and he’ll close the deal soon.

But it will never happen.

The reason it never happens is because it’s not a real YES. It was a fake YES by the customer to escape.

We are taught to be polite. It’s considered bad manners to bluntly reject someone who’s eager to please, even a salesperson. We learnt not to burn bridges, just in case one day we may change our mind.

So we reject pitches by giving people seemingly positive feedback and approval. Not too much, just enough to end the conversation.

We kick the can down the road. We want to soften the blow.

When I meet a customer, I look for a NO.

I notice that people often know what they don’t want far better than what they want. They articulate NO much more clearly than YES. A NO is always concrete.

I met a client recently with my salesperson. His pitch was energetic and convincing, covering all the bases from data quality, different asset classes to templates for option strategies.

The client was nodding and smiling. He said he would love to try it if he had more time.

I could see that this was not a real YES. I wanted to get from him a concrete NO.

I said: “Even with all these tools, it’s not easy to research option strategies, is it?”

His eyes lit up: “It’s so hard. We have to focus on ideas that make money. I don’t know the option space that well.”

I got it. I admired his courage to admit that the bottleneck wasn’t necessarily the product, but his own expertise. 

I offered a suggestion: “We recently launched an AI agent that can read the description of a trade idea in English and automatically implement it in Python. Will this help you test more ideas?”

His body language softened and he asked: “How do you price it?” 

In my experience, the only reliable sign of a real YES is when a client asks about the price.

People love saying NO because it makes them feel safe and grounded.

Often we just need to encourage them to voice NO in order to dig deeper into specific issues and topics.

That may be the only way to get to a real YES.

* A powerful black rhiNO in the Kruger National Park, South Africa, 2018

SigTech Origin Story

Clients often ask me why we named our company SigTech.

I explain the first part of our name has two origins: “Sig” is short for signals. We are in the business of providing technology so investors can find signals in the market.

It is also a nod to our history: “SIG” was the acronym for Systematic Investment Group, the team I led when I worked at the hedge fund Brevan Howard.

When “Sig” is combined with “Tech” it explains our mission: to provide leading technology to help investors distinguish “signals” from the “noise”.

The idea of the business was first conceived while I was hiking Inca trails to reach Machu Picchu in late 2017.

In 2019, we officially started out offering services to buy-side portfolio managers at hedge funds, pension funds, and sovereign wealth funds. We are now expanding that effort by leveraging generative AI to make those advanced decision-making tools available to anyone in capital markets.

I’m passionate about democratizing access to data and information in part because I came to the world of finance unexpectedly.

After completing my undergraduate degree in electrical and electronics engineering at China’s Shanghai Jiaotong University in 2003, I began doctoral studies in computer science at the University of Cambridge.

That gave me the opportunity to help research and develop Xen, the virtual machine monitor that ushered in the era of cloud computing when it enabled Amazon to build AWS.

In 2007, after my PhD studies, I went to work at Barclays Capital in quantitative investment strategy trading, and in 2015, Brevan Howard where I later became the chief investment officer running the Systematic Investment Group.

Brevan Howard puts an emphasis on intellectual honesty. The culture encourages employees to analyze trades to determine how much of our success was due to luck versus skill. In the past two decades, the firm has generated over $30 billion in returns for investors.

In February 2019, Brevan Howard spun off SigTech as a fintech business supporting other firms making data-driven investments.

This process of distilling data into signals is challenging. You must overcome a high degree of background noise in the data. But this distillation is core to the investment process.

Once you find a signal, you need to determine if it will accurately predict an outcome significantly better than the market. You also have to decide if you can structure trades to monetize your prediction so that your upside is greater than the downside.

Solving those problems is where generative AI comes in, and it’s a game-changer.

At SigTech we expect the technology we are building to change how decisions are made in capital markets.



Creative Capital: AI Will Change Finance, Investing & Trading Forever I SigTech CEO Bin Ren

I sat down with Eddie Donmez, the founder and CEO of Creative Capital, to talk about my career and AI at SigTech:

In the latest Creative Capital interview, Eddie Donmez sits down with SigTech founder and CEO, Bin Ren. Bin has a crazy story. Before founding SigTech, he worked on the code that became what we know today as Amazon Web Services (AWS) and worked for one of the most famous hedge funds in the world Brevan Howard. After working closely with Alan Howard, he spun SigTech (Systematic Investing Group) out of Brevan to provide cutting-edge technology to hedge funds, sovereign wealth funds, and endowments. He has now developed a game-changing generative AI tool for all investors - SigTechAI. It offers access to premium financial data, performs accurate portfolio analysis, and endless tools for investing. No math. No code. In the interview, we discuss the implications of AI in finance and how it will prove to be a game changer for finance, investing, wealth management, and more.  

This is a MUST LISTEN for any investor or student of the game.

You can watch the whole interview here on YouTube.

AI Founders Podcast: The Future of Trading: How Advanced AI Models Could Transform Market Strategies

I had one of my favourite conversations with Jay Singh, CEO and founder of Casper Studios, on his AI Founders Podcast:

In this episode, I spoke with Bin Ren, CEO and founder of SigTech, a fintech platform that combines AI and quantitative analysis for market professionals. Bin shared his journey from Shanghai to London, discussing how cultural experiences shaped his entrepreneurial mindset and the importance of balancing exploration and exploitation in creativity.

We also explored the transformative potential of generative AI in finance, particularly its ability to integrate numerical analysis with textual insights. Bin emphasized the need for intellectual honesty in assessing both successes and failures, and he articulated a vision for using advanced reasoning models to enhance trade structuring, aiming to revolutionize traders' strategies. This conversation highlighted the exciting intersection of technology and finance.

You can listen to it here, on Spotify or Apple Podcasts.

Numerix Trading Tomorrow: The Impact of AI on Capital Markets with Bin Ren

I had the pleasure of speaking with James Jockle, EVP and CMO of Numerix on the Trading Tomorrow podcast:

Unlock the future of finance with Bin Ren, Founder & CEO of SigTech, as he reveals the transformative potential of AI in capital markets. Discover how AI is revolutionizing financial decision-making processes by enhancing productivity tools for professionals in investment management, trading, and risk management. Learn about the critical role of a robust data foundation in building AI-driven systems and the intricate stages of pre-training and post-training large language models. Bin shares practical examples to illustrate how AI can swiftly process and summarize complex information, potentially altering how financial decisions are made.

You can listen to it on Buzzsprout, or Spotify or Apple Podcasts.

Citi Good Things Happen, E8: Transforming Finance with AI

I had the pleasure to sit down with Alex Miller from Citi’s Global Insights team and Ronit Ghose, Citi’s Head of Future Finance to discuss the emerging role of artificial intelligence in global banking on Good Things Happen podcast:

In the series finale of Good Things Happen, we joined Citi’s 11th annual Digital Money Symposium in London which took place on 20th June, to be inspired by thought leaders across the financial services ecosystem, discussing ‘the future of finance and money’.

Guest host Alex Miller, from Citi’s Global Insights team is joined by Bin Ren, Founder & CEO of Sig Tech and Ronit Ghose, Citi’s Head of Future Finance to discuss the emerging role of artificial intelligence in global banking.

Bin Ren tells us about the autonomous AI agents his company is building, including those able to pass the Chartered Finance Analyst exam at 90 percentile performance. He describes the opportunities he sees to use AI to scale the industry, boost productivity and enhance jobs.

Ronit Ghose, joins the podcast to talk through AI and the considerations and regulations being put in place to avoid and mitigate risks such as deep fakes and AI hallucinations.

You can listen to it here.