{"id":26147,"date":"2023-06-02T20:44:22","date_gmt":"2023-06-02T18:44:22","guid":{"rendered":"https:\/\/quantpedia.com\/?p=26147"},"modified":"2025-06-04T14:29:41","modified_gmt":"2025-06-04T12:29:41","slug":"in-sample-vs-out-of-sample-analysis-of-trading-strategies","status":"publish","type":"post","link":"https:\/\/vvv.quantpedia.com\/es\/in-sample-vs-out-of-sample-analysis-of-trading-strategies\/","title":{"rendered":"In-Sample vs. Out-Of-Sample Analysis of Trading Strategies"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong><a href=\"https:\/\/www.vox.com\/future-perfect\/21504366\/science-replication-crisis-peer-review-statistics\">Science has been in a \u201creplication crisis\u201d for more than a decade.<\/a> Researchers have discovered, over and over, that lots of findings in fields like psychology, sociology, medicine, and economics don\u2019t hold up when other researchers try to replicate them. There are many interesting questions of philosophy of science, for example: Is the problem just that we test for \u201cstatistical significance\u201d \u2014 the likelihood that similarly strong results could have occurred by chance \u2014 in a nuance-free way? Is it that null results (that is when a study finds no detectable effects) are ignored while positive ones make it into journals? Simply said: many published studies cannot be replicated.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>But what does it mean to us, investors and traders? We, here at Quantpedia, try to present you <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-it-works\/\">academic research in a digestible form<\/a> for the person that is not used to going rigorously through myriads of papers written in &#8220;academese&#8221; and often hard to understand true applicable meaning of it.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><strong>So, is there any \u201cedge\u201d in purely academic-developed trading strategies and investment approaches after publishing, or will they perish shortly after becoming public? After some time, we are about to\u00a0<\/strong><a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-do-investment-strategies-perform-after-publication\/\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>revisit our own concept<\/strong><\/a><strong>\u00a0and test the out-of-sample decay. But this time, we have hard data &#8211; our regularly updated\u00a0<\/strong><a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/screener\/?screener_tab=screener-classic&amp;FilterQuantConnect=With+QuantConnect+Code&amp;FilterPeriodicUpdates=Yes\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>database of replicated quant strategies<\/strong><\/a>.<\/strong><\/p>\n\n\n\n<center>Video summary:<\/center>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<figure class=\"wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio responsive-video wp-embed-aspect-4-3\"><div class=\"wp-block-embed__wrapper\">\n<iframe title=\"In-Sample vs Out-Of-Sample Analysis of Trading Strategies - Quantpedia Explains (Trading Strategies)\" width=\"800\" height=\"450\" src=\"https:\/\/www.youtube.com\/embed\/NW6Pgw4wOv4?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<h1 class=\"wp-block-heading\">Introduction<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">When an anomaly is discovered and a strategy built around it is shared, it often leads to concerns that the anomaly might be arbitraged away and potentially turning unprofitable in investors&#8217; portfolios. However, <strong>investment strategies typically remain profitable after publication<\/strong>, although there is a decrease in profitability. However, the returns do not instantly weaken; a significant part of the return remains even after the strategy becomes widely known. A study conducted by <a href=\"https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=2156623\">McLean and Pontiff<\/a> found that portfolio returns were 26% lower out-of-sample and 58% lower during the five years post-publication, indicating that investors are aware of academic publications and learn about mispricing. However, the diminishing of returns happens gradually over time, and even after five years, a remarkable part of an anomaly\u2019s return is preserved. The known anomaly often transforms into a &#8216;smart beta factor&#8217; and can still be profitably used within a diversified portfolio. It&#8217;s also noted that the publication process of academic papers can take one or two years, and during this time, practitioners can extract ideas from working papers to gain an advantage.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A recent paper by Jensen, Kelly &amp; Pedersen:&nbsp;<a href=\"https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=3781319\" target=\"_blank\" rel=\"noreferrer noopener\">Is There a Replication Crisis in Finance?<\/a>&nbsp;also discusses the replication of trading strategies described in academic research. The researchers found that over 80% of US equity factors remained significant even after making adjustments for consistent and more implementable factor construction while still preserving the original signal. In addition, the same quality and quantity of behavior were observed across 153 factors in 93 countries, suggesting a high degree of external validity in factor research. That&#8217;s really not a bad result &#8230;<\/p>\n\n\n\n<p><img fetchpriority=\"high\" decoding=\"async\" class=\"aligncenter size-full wp-image-27375\" src=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/wp-content\/uploads\/2023\/06\/Picture-287-Alpha-of-trading-strategies-from-academic-research.png\" alt=\"\" width=\"912\" height=\"631\" srcset=\"https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/06\/Picture-287-Alpha-of-trading-strategies-from-academic-research.png 912w, https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/06\/Picture-287-Alpha-of-trading-strategies-from-academic-research-300x208.png 300w\" sizes=\"(max-width: 912px) 100vw, 912px\" \/><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Not too much disturbing are also findings from&nbsp;<a href=\"https:\/\/wp.lancs.ac.uk\/fofi2018\/files\/2018\/03\/FoFI-2018-0174-Sebastian-M%C3%BCller2.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">Heiko Jacobs and Sebastian Muller in their Anomalies across the globe: Once public, no longer existent?<\/a>&nbsp;Motivated by McLean and Pontiff (2016) (which we will mention later), they studied the pre- and post-publication return predictability of 231 cross-sectional anomalies in 39 stock markets. Based on more than two million anomaly country months, their result is that the United States is the only country with a reliable post-publication decline in long\/short returns. This might provide invaluable insights into the modeling of the &#8220;longevity&#8221; of strategies performed on these markets and &#8220;life expectancy&#8221; until strategy\/anomaly &#8220;dies.&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The last research paper that we would like to mention is that of&nbsp;<a href=\"https:\/\/ssrn.com\/abstract=3845928\" target=\"_blank\" rel=\"noreferrer noopener\">Falck, Rej, and Thesmar (2021)<\/a>, which examines the hypothesis of ex-ante characteristics empirically predicting the out-of-sample (they define out-of-sample as the post-publication period) drop in risk-adjusted performance of published stock anomalies. Their final conclusion is that every year, the Sharpe decay of newly-published factors increases by around 5% (which is not so much).<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">Data &amp; Methodology<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">We have collected all data about strategies from our backtests (the ones you see in the end section of most of the Quantpedia Strategies found in our&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/screener\" target=\"_blank\" rel=\"noreferrer noopener\">Screener<\/a>) that were done in&nbsp;<a href=\"https:\/\/www.quantconnect.com\/?utm_source=sdkfjssdfgsdm5qwlks8323dslkdfjsx246s30dlsaaslgk&amp;ref=radovanvojtko\" target=\"_blank\" rel=\"noreferrer noopener\">QuantConnect<\/a>. Next, we diligently get information about when the data from the source papers end (end of data set testing period in the source academic paper). This bases the intersection between out-of-sample and in-sample for our backtests. Next, we divide these data into those two sets and compare and analyze them.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s now show you our selected methodology for this experimental validation of our backtests on one example:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s use our first-ever encyclopedic entry, the evergreen <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/strategies\/asset-class-trend-following\/\">Asset Class Trend-Following<\/a> strategy, based on Mebane Faber&#8217;s paper <a href=\"https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=962461\">A Quantitative Approach to Tactical Asset Allocation<\/a>. End of the backtesting period from paper sets point of intercept between datasets (the year after the hyphen) (on picture highlighted with green):<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/strategies\/asset-class-trend-following\/\"><img decoding=\"async\" width=\"1024\" height=\"446\" src=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/wp-content\/uploads\/2023\/05\/Screenshot-Asset-Class-Trend-Following-QuantPedia-Asset-Class-Trend-Following.cleaned-1024x446.png\" alt=\"\" class=\"wp-image-26652\" srcset=\"https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/05\/Screenshot-Asset-Class-Trend-Following-QuantPedia-Asset-Class-Trend-Following.cleaned-1024x446.png 1024w, https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/05\/Screenshot-Asset-Class-Trend-Following-QuantPedia-Asset-Class-Trend-Following.cleaned-300x131.png 300w, https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/05\/Screenshot-Asset-Class-Trend-Following-QuantPedia-Asset-Class-Trend-Following.cleaned.png 1366w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">And this is being that year shown in QUANTCONNECT backtest. Up until the last day of 2008, we consider it <em>in-sample<\/em> backtest results, onwards from the first trading day of 2009 <em>out-of-sample<\/em> backtest results.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img decoding=\"async\" width=\"1024\" height=\"485\" src=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/wp-content\/uploads\/2023\/06\/Picture-288-In-vs-Out-of-Sample-trading-1024x485.png\" alt=\"\" class=\"wp-image-27380\" srcset=\"https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/06\/Picture-288-In-vs-Out-of-Sample-trading-1024x485.png 1024w, https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/06\/Picture-288-In-vs-Out-of-Sample-trading-300x142.png 300w, https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/06\/Picture-288-In-vs-Out-of-Sample-trading.png 1179w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">From all 868&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/screener\/page\/44\/\" target=\"_blank\" rel=\"noreferrer noopener\">strategies<\/a>&nbsp;at the moment of analysis (at the beginning of&nbsp;<em>May 2023<\/em>), we have in-house backtested&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/screener\/?screener_tab=screener-classic&amp;FilterQuantConnect=With+QuantConnect+Code\" target=\"_blank\" rel=\"noreferrer noopener\"><em>671<\/em><\/a>&nbsp;out of them. Out of those,&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/screener\/?screener_tab=screener-classic&amp;FilterQuantConnect=With+QuantConnect+Code&amp;FilterPeriodicUpdates=Yes\" target=\"_blank\" rel=\"noreferrer noopener\"><em>417<\/em><\/a>&nbsp;were maintained and updated on a monthly basis, and those were therefore included in the analysis (and they can also be further used in&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/pricing-pro\/\" target=\"_blank\" rel=\"noreferrer noopener\">Quantpedia Pro<\/a>&#8216;s Portfolio Manager and analytical tools).&nbsp;Further, 62 of them were excluded; we did not have data to get back into the period ending from the paper (we started backtesting from a date later than the end date of the data sample indicated in the paper). This concludes the data preprocessing part. So, we are left with <strong><em>355<\/em><\/strong> strategies for further analysis.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The main analyzed core interest is the&nbsp;<em>Sharpe Ratio<\/em>, which we conclude is the most suitable measure to compare in- vs. out-of-sample results in our data sample. Our&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/charts\/\" target=\"_blank\" rel=\"noreferrer noopener\">database covers all main asset classes<\/a>&nbsp;(equities, bonds, commodities, cryptos). Therefore, it would be meaningless to draw conclusions from annualized return measures when our data sample contains highly volatile crypto strategies and, at the same time, low volatile fixed-income strategies. The Sharpe ratio measure is great in our case because we scale annualized returns by yearly volatility, which helps to analyze risk-adjusted returns.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So, now we have both:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>in-sample dataset (usually <span class=\"typ\" style=\"white-space: pre-wrap;\"><span class=\"typ\" style=\"white-space: pre-wrap;\"><code>StartDate<\/code> from QP until the end of <\/span><\/span><span style=\"text-decoration: underline;\">Backtest period from source paper<\/span>),<\/li>\n\n\n\n<li>out-of-sample dataset (<span class=\"typ\"><span class=\"typ\">end of <\/span><\/span>Backtest period from source paper until the end of QP backtest; in case of one-time-off code with limited data, is fixed, in case of recurring and updating code, always dynamic and adjusted to current month);<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">We calculated <em>CAR p.a.<\/em> and <em>Volatility p.a.<\/em>, which division gave us <em>Sharpe Ratio<\/em>s for in-sample and out-of-sample for each included strategy.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">Results and their Interpretation<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Now, let&#8217;s move on. As a measure of average, we have chosen the arithmetic mean. The&nbsp;<a href=\"https:\/\/en.wikipedia.org\/wiki\/Median\" target=\"_blank\" rel=\"noreferrer noopener\">median<\/a>&nbsp;is the value separating the higher half from the lower half of a data sample and is not skewed by a small proportion of extremely large or small values, providing a better representation of the center. It is the main reason why we include it in our analysis too. The following table depicts the average and median Sharpe Ratios among the in- and out-of-sample data:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><br \/>\n<table class=\" aligncenter\" style=\"width: 75%; height: 115px;\" align=\"center\"><caption><span style=\"font-size: 18pt;\">Tab. 1 Comparison of Sharpe Ratios of backtested strategies<\/span><\/caption>\n<tbody>\n<tr style=\"height: 23px;\">\n<td style=\"width: 33.3333%; height: 23px;\"><strong>Sharpe Ratio (of)<br \/><\/strong><\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><strong>average<\/strong><\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><strong>median<\/strong><\/td>\n<\/tr>\n<tr style=\"height: 23px;\">\n<td style=\"width: 33.3333%; height: 23px;\">in-sample results<\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><span data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:1.5736178848076665}\" data-sheets-userformat=\"{&quot;2&quot;:14337,&quot;3&quot;:{&quot;1&quot;:2,&quot;2&quot;:&quot;0.000&quot;,&quot;3&quot;:1},&quot;14&quot;:{&quot;1&quot;:3,&quot;3&quot;:1},&quot;15&quot;:&quot;Arial&quot;,&quot;16&quot;:10}\" data-sheets-formula=\"=AVERAGE(R[0]C[-9]:R[357]C[-9])\">1.574<\/span><\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><span data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:1.1802304042226328}\" data-sheets-userformat=\"{&quot;2&quot;:14337,&quot;3&quot;:{&quot;1&quot;:2,&quot;2&quot;:&quot;0.000&quot;,&quot;3&quot;:1},&quot;14&quot;:{&quot;1&quot;:3,&quot;3&quot;:1},&quot;15&quot;:&quot;Arial&quot;,&quot;16&quot;:10}\" data-sheets-formula=\"=MEDIAN(R[0]C[-10]:R[357]C[-10])\">1.180<\/span><\/td>\n<\/tr>\n<tr style=\"height: 23px;\">\n<td style=\"width: 33.3333%; height: 23px;\">out-of-sample results<\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><span data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:1.0485029952356257}\" data-sheets-userformat=\"{&quot;2&quot;:14337,&quot;3&quot;:{&quot;1&quot;:2,&quot;2&quot;:&quot;0.000&quot;,&quot;3&quot;:1},&quot;14&quot;:{&quot;1&quot;:3,&quot;3&quot;:1},&quot;15&quot;:&quot;Arial&quot;,&quot;16&quot;:10}\" data-sheets-formula=\"=AVERAGE(R[-1]C[-2]:R[356]C[-2])\">1.049<\/span><\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><span data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:0.6620762912668194}\" data-sheets-userformat=\"{&quot;2&quot;:14337,&quot;3&quot;:{&quot;1&quot;:2,&quot;2&quot;:&quot;0.000&quot;,&quot;3&quot;:1},&quot;14&quot;:{&quot;1&quot;:3,&quot;3&quot;:1},&quot;15&quot;:&quot;Arial&quot;,&quot;16&quot;:10}\" data-sheets-formula=\"=MEDIAN(R[-1]C[-3]:R[356]C[-3])\">0.662<\/span><\/td>\n<\/tr>\n<tr style=\"height: 23px;\">\n<td style=\"width: 33.3333%; height: 23px;\">\u0394 (in-out)<\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><span data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:0.5251148895720408}\" data-sheets-userformat=\"{&quot;2&quot;:14399,&quot;3&quot;:{&quot;1&quot;:2,&quot;2&quot;:&quot;0.000&quot;,&quot;3&quot;:1},&quot;4&quot;:{&quot;1&quot;:2,&quot;2&quot;:16043212},&quot;5&quot;:{&quot;1&quot;:[{&quot;1&quot;:2,&quot;2&quot;:0,&quot;5&quot;:{&quot;1&quot;:0}},{&quot;1&quot;:0,&quot;2&quot;:0,&quot;3&quot;:3},{&quot;1&quot;:1,&quot;2&quot;:0,&quot;4&quot;:1}]},&quot;6&quot;:{&quot;1&quot;:[{&quot;1&quot;:2,&quot;2&quot;:0,&quot;5&quot;:{&quot;1&quot;:0}},{&quot;1&quot;:0,&quot;2&quot;:0,&quot;3&quot;:3},{&quot;1&quot;:1,&quot;2&quot;:0,&quot;4&quot;:1}]},&quot;7&quot;:{&quot;1&quot;:[{&quot;1&quot;:2,&quot;2&quot;:0,&quot;5&quot;:{&quot;1&quot;:0}},{&quot;1&quot;:0,&quot;2&quot;:0,&quot;3&quot;:3},{&quot;1&quot;:1,&quot;2&quot;:0,&quot;4&quot;:1}]},&quot;8&quot;:{&quot;1&quot;:[{&quot;1&quot;:2,&quot;2&quot;:0,&quot;5&quot;:{&quot;1&quot;:0}},{&quot;1&quot;:0,&quot;2&quot;:0,&quot;3&quot;:3},{&quot;1&quot;:1,&quot;2&quot;:0,&quot;4&quot;:1}]},&quot;14&quot;:{&quot;1&quot;:3,&quot;3&quot;:1},&quot;15&quot;:&quot;Arial&quot;,&quot;16&quot;:10}\" data-sheets-formula=\"=R[1]C[0]-R[2]C[0]\">-0.525<\/span><\/td>\n<td style=\"width: 33.3333%; height: 23px;\"><span data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:0.5181541129558134}\" data-sheets-userformat=\"{&quot;2&quot;:14337,&quot;3&quot;:{&quot;1&quot;:2,&quot;2&quot;:&quot;0.000&quot;,&quot;3&quot;:1},&quot;14&quot;:{&quot;1&quot;:2,&quot;2&quot;:0},&quot;15&quot;:&quot;Arial&quot;,&quot;16&quot;:10}\" data-sheets-formula=\"=R[1]C[0]-R[2]C[0]\">-0.518<\/span><\/td>\n<\/tr>\n<tr style=\"height: 23px;\">\n<td style=\"width: 33.3333%; height: 23px;\">\u0394% (in\/out)<\/td>\n<td style=\"width: 33.3333%; height: 23px;\" data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:0.3336991112275152}\" data-sheets-numberformat=\"{&quot;1&quot;:3,&quot;2&quot;:&quot;0.00%&quot;,&quot;3&quot;:1}\" data-sheets-formula=\"=R[-11]C[0]\/R[-10]C[0]\">-33.37%<\/td>\n<td style=\"width: 33.3333%; height: 23px;\" data-sheets-value=\"{&quot;1&quot;:3,&quot;3&quot;:0.43902793141234087}\" data-sheets-numberformat=\"{&quot;1&quot;:3,&quot;2&quot;:&quot;0.00%&quot;,&quot;3&quot;:1}\" data-sheets-formula=\"=R[-11]C[0]\/R[-10]C[0]\">-43.90%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">As per Tab. 1, Sharpe Ratio for out-of-sample results is worse and deteriorated by&nbsp;<strong>33% (on average) or 44% (median strategy)<\/strong>. These results are totally in line and consistent with findings in the previously mentioned research papers and&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-do-investment-strategies-perform-after-publication\/\" target=\"_blank\" rel=\"noreferrer noopener\">previous blog post we wrote about this topic<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Following is the figure depicting the distribution (histogram) of <em>Sharpe Ratio<\/em>s both in- and out-of-sample:<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/wp-content\/uploads\/2023\/05\/Sharpe_in.cleaned.png\" alt=\"\"\/><\/figure>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img decoding=\"async\" src=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/wp-content\/uploads\/2023\/05\/Sharpe_out.cleaned.png\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">As we can see, the distribution of results follows &#8220;quasi-normal distribution&#8221;, where you can mainly expect your strategy to have a Sharpe ratio in interval <em>(-1, 3)<\/em>. And it is both for out-of-sample and in-sample results. In-sample results are, of-course, faring a bit better. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Another interesting observation is that both in-sample and out-of-sample results seem to be skewed positively &#8211; fat tail on the right side (but the out-of-sample histogram right tail is a little less thick). This is a very interesting finding and speaks in favor of stringent risk management if we employ a portfolio of multiple strategies. It seems strategies seem to deteriorate an out-of-sample, but we have some really strong positive outliers; therefore, it makes sense to cut the risk budget to strategies that do not perform well out-of-sample and let profit runs in those that profit well. Those results may give some validity to the idea of&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/a-deeper-look-into-factor-momentum\/\" target=\"_blank\" rel=\"noreferrer noopener\">Factor Momentum<\/a>&nbsp;&#8211; it may be a good idea to increase weight for strategies that recently performed well.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But let&#8217;s move on. This is the picture showing how all included strategies behaved during the 10-year window, the last 5 years of in-sample, and the first 5 years in out-of-sample:<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"640\" height=\"480\" src=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/wp-content\/uploads\/2023\/05\/kiWipv.cleaned.png\" alt=\"\" class=\"wp-image-26529\" srcset=\"https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/05\/kiWipv.cleaned.png 640w, https:\/\/vvv.quantpedia.com\/wp-content\/uploads\/2023\/05\/kiWipv.cleaned-300x225.png 300w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">On X-axis, we have an in-sample period (-5 to 0), year zero is the threshold year, and from year 0 to 5 is the performance in the out-of-sample period. Y-axis shows an appreciation of 1 USD during the previously mentioned time. All strategies in year 0 start at 1 USD. The black line depicts the (simple arithmetic) average multiplier of returns when we mix all strategies in an equally-weighted portfolio. Our point here is to show that performed strategies have positive expectancy, but dispersion in performance (in- and also out-of-sample) is really significant.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Assuming we build a portfolio incorporating all backtested strategies weighted equally (black line), we also calculated what would be signal loss in performance after publication. Following our indicated approach, if you start using each strategy on the date when it ends being backtested by paper and weight your portfolio equally, we found that the out-of-sample portfolio would on average yield approximately 4\/5 of in sample performance. Once again, individual performance decay among strategies varies a lot.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The performance decay in the portfolio approach is lower than performance decay in individual strategies. The reason for that is the low correlation among strategies (in- and also out-of-sample).<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">Conclusion<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Undoubtedly,&nbsp;<strong><em>Sharpe Ratio<\/em><\/strong><em>s<\/em>&nbsp;worsen during a lifetime after forming various trading strategies based on beforehand not-know market anomalies. According to our analysis, it is reasonable to expect a Sharpe ratio degradation by 1\/3 or by 1\/2 against the in-sample period. This should not be, by any means, discouraging, but rather a positive finding that one should embrace and prepare to account for less pleasant sides of investing and trading, considering reported and expected returns and unexpected deviations from them.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Our results are in line with the current academic consensus. As in our previously mentioned&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-do-investment-strategies-perform-after-publication\/\" target=\"_blank\" rel=\"noreferrer noopener\">blog post from 2020<\/a>,&nbsp;<a href=\"https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=2156623\" target=\"_blank\" rel=\"noreferrer noopener\">McLean and Pontiff (2016)<\/a>&nbsp;find that portfolio returns performing various trading strategies based on a statistically significant sample of different variables and factors covering the representable number of anomalies in total are 26% lower out-of-sample and 58% lower post-publication, which is not far away from our numbers.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What could be the partial solution for the performance decay? Preliminary findings suggest that factor momentum could be the right answer. Fat-tailed financial distributions are known to be good targets to exploit by momentum\/trend-following rules. So momentum\/trend overlay on a portfolio of strategies can be a good idea. Alternatively, other price-based factor overlays (low volatility, MIX, MAX, etc.) may also be considered, as we reviewed in our series of blog posts in which we investigated <strong>Social Trading Multi-Strategy (<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/community-alpha-of-quantconnect-part-1-following-numerous-quantitative-strategies\/\" target=\"_blank\" rel=\"noreferrer noopener\">1<\/a>,&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/community-alpha-of-quantconnect-part-2-social-trading-factor-strategies\/\" target=\"_blank\" rel=\"noreferrer noopener\">2<\/a>,&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/community-alpha-of-quantconnect-part-3-adjusted-social-trading-factor-strategies\/\" target=\"_blank\" rel=\"noreferrer noopener\">3<\/a>, and&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/community-alpha-of-quantconnect-part-4-composite-social-trading-multi-factor-strategy\/\" target=\"_blank\" rel=\"noreferrer noopener\">4<\/a>).<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Author:<br>Cyril Dujava, Quant Analyst, Quantpedia<\/strong><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"block-854363cc-8450-4dc0-a06a-c737766e9431\"><strong>Are you looking for more strategies to read about? <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/sign-up-for-our-newsletter\/\">Sign up for our newsletter<\/a> or visit our <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/blog\/\">Blog<\/a> or <a href=\"http:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/Screener\">Screener<\/a><\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"block-65925002-6290-4d3b-b5cd-f3a277851ec8\"><strong>Do you want to learn more about Quantpedia Premium service? Check <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-it-works\/\">how Quantpedia works<\/a>, <a href=\"http:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/Home\/About\">our mission<\/a> and <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/pricing\/\">Premium pricing offer<\/a>.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"block-34bf63ae-5a22-40a3-aeb4-769374e833d8\"><strong>Do you want to learn more about Quantpedia Pro service? Check its <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-it-works\/quantpedia-pro\/\">description<\/a>, watch <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-it-works\/quantpedia-explains\/\">videos<\/a>, review <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-it-works\/quantpedia-pro-reports\/\">reporting capabilities<\/a> and visit our <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/pricing-pro\/\">pricing offer<\/a>.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"block-21942b3a-14d9-4c0f-b8ef-04d64675e253\"><strong>Are you looking for historical data or backtesting platforms? Check our list of&nbsp;<a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/links-tools\/?category=algo-trading-discounts\">Algo Trading Discounts<\/a><\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Would you like free access to <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/pricing\/\" title=\"\">our services<\/a>? Then, <a href=\"https:\/\/lightspeed.com\/lp\/quantpedia-lightspeed-financial-services-group-one-free-year-promotion\" title=\"\">open an account with Lightspeed<\/a> and enjoy one year of Quantpedia Premium at no cost.<\/strong><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"block-4c45d6c9-c8dd-4283-8743-bf573cfa4d45\"><strong>Or follow us on:<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"block-476e95ed-31a5-4c4d-b701-5203f9fb2e24\"><strong>Facebook <a href=\"https:\/\/www.facebook.com\/groups\/quantstrategies\">Group<\/a>, Facebook <a href=\"https:\/\/www.facebook.com\/quantpedia\/\">Page<\/a>, <a href=\"https:\/\/twitter.com\/quantpedia\">Twitter<\/a>, <a href=\"https:\/\/www.linkedin.com\/company\/quantpedia\">Linkedin<\/a>, <a href=\"https:\/\/quantpedia.medium.com\/\">Medium<\/a> or <a href=\"https:\/\/www.youtube.com\/channel\/UC_YubnldxzNjLkIkEoL-FXg\">Youtube<\/a><\/strong><\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p><strong><a href=\"https:\/\/www.vox.com\/future-perfect\/21504366\/science-replication-crisis-peer-review-statistics\"><strong>Science has been in a \u201creplication crisis\u201d for more than a decade.<\/strong><\/a> But what does it mean to us, investors and traders? Is there any \u201cedge\u201d in purely academic-developed trading strategies and investment approaches after publishing, or will they perish shortly after becoming public? After some time, we will revisit our <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/how-do-investment-strategies-perform-after-publication\/\"><strong>older blog<\/strong><\/a> on this theme and test the out-of-sample decay of trading strategies. But this time, we have hard data &#8211; our regularly updated <a href=\"https:\/\/\\\/\\\/new-fmhwbzh6ghd9hede.swedencentral-01.azurewebsites.net\/screener\/?screener_tab=screener-classic&#038;FilterQuantConnect=With+QuantConnect+Code&#038;FilterPeriodicUpdates=Yes\"><strong>database of replicated quant strategies<\/strong><\/a>.<\/strong><\/p>","protected":false},"author":25210,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[197,163,147,159,146],"class_list":["post-26147","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-diversification","tag-factor-allocation","tag-factor-investing","tag-own-research","tag-smart-beta"],"_links":{"self":[{"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/posts\/26147","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/users\/25210"}],"replies":[{"embeddable":true,"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/comments?post=26147"}],"version-history":[{"count":0,"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/posts\/26147\/revisions"}],"wp:attachment":[{"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/media?parent=26147"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/categories?post=26147"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vvv.quantpedia.com\/es\/wp-json\/wp\/v2\/tags?post=26147"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}