Research interview, close to the centre of the job.
The backtest that will not survive
The question
You tried two hundred signal variants and kept the best. Its backtest Sharpe is 2.5. What do you actually believe?
Have a go before you open the solution. If you are stuck after a few minutes, the first paragraph below is usually enough of a nudge.
Show the worked solution
The answer
Very little, until you adjust for the number of trials. The maximum of many noisy estimates is biased high.
With 200 independent trials on pure noise, the best in-sample Sharpe is expected to be around sqrt(2 ln 200), roughly 3.2 standard errors above zero.
So a best-of-200 Sharpe of 2.5 is entirely consistent with having no edge at all.
The corrections are to report the deflated Sharpe, hold out data you never looked at, and record the true number of trials, including the ones you ran informally.
The answer interviewers want is the trials count. Candidates who defend the 2.5 without mentioning it fail this question.
What it is testing
Multiple-testing awareness, the most common real failure in research work.
Try these next
The third correlation
StandardA and B are 0.9 correlated, B and C are 0.9 correlated. What range is possible for the correlation of A and C?
Why the top performers disappoint
Warm-upA fund's best traders this year are usually worse next year. Does that mean the firm is failing to develop them?
How long to trust a Sharpe ratio
HardA strategy shows a Sharpe of 2 over six months. How much should that convince you?
Is this series stationary?
StandardWhy does stationarity matter before you model a price series, and what do you do about it?
Preparing properly
Live opportunities
Interviewing soon? See who is hiring
Open vacancies we track at quant firms, grouped by role and city. Adverts are anonymised.
Hiring for these roles? Brief the desk on a hire or check your fit for a role.
Research desk
Speak to a quant market specialist
Ask what a firm really pays, how a desk hires, or where your profile sits. One reply from the research desk, in confidence, no obligation.