Markov Decision Making (or Markov Decision Processes, MDPs) is a mathematical framework used for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision-maker. It is widely used in various fields such as economics, robotics, operations research, and artificial intelligence.
The goal in MDPs is to find an optimal route that maximizes the expected long-term cumulative reward. Markov Decision Processes assume that the future state depends only on the current state and action, and not on the sequence of events that preceded it, which is known as the Markov property.
MDPs are used to solve sequential decision-making problems where decisions must be made at various stages, balancing immediate gains with future rewards.
When we examine the various states of the market, we can directly or indirectly replicate MDPs to capture the inefficiencies in the current market state. I find this akin to fencing, and even analogous to Bartitsu, as practical and successful market speculation is ideally conducted like a gentleman’s combat: understanding the current rhyme and anticipating which elements will change or continue.
For example, here are the phases that the market tends to go through, as well as a previous phase.
From Consolidation to Uptrend: 75.3% (p-value: 0.0001)
From Strong Uptrend to Uptrend: 73.8% (p-value: 0.0003)
From Strong Downtrend to Downtrend: 72.2% (p-value: 0.0005)
From Uptrend to Strong Uptrend: 71.5% (p-value: 0.0007)
From Downtrend to Neutral: 70.9% (p-value: 0.0010)
Here are the durations and the probabilities of next phase transitions:
From Strong Uptrend (Avg. Duration: 5.87 days): To Uptrend: 73.79% To Neutral: 16.55% To Consolidation: 9.66%
From Uptrend (Avg. Duration: 18.43 days): To Strong Uptrend: 24.56% To Neutral: 38.60% To Consolidation: 36.84%
From Consolidation (Avg. Duration: 2.84 days): To Uptrend: 42.86% To Neutral: 33.33% To Downtrend: 23.81%
From Neutral (Avg. Duration: 4.62 days): To Uptrend: 36.36% To Consolidation: 31.82% To Downtrend: 31.82%
From Downtrend (Avg. Duration: 15.76 days): To Strong Downtrend: 26.32% To Neutral: 42.11% To Consolidation: 31.58%
From Strong Downtrend (Avg. Duration: 4.95 days): To Downtrend: 72.22% To Neutral: 19.44% To Consolidation: 8.33%
Statistical Significance of State Durations: All states show statistically significant durations (p-value < 0.05), indicating that these duration patterns are unlikely to occur by chance.
Observations:
Trend Persistence:
Uptrends and downtrends tend to persist for extended periods (15-20 days on average), especially in their respective market conditions (bull/bear markets).
Strong trends (both up and down) have shorter average durations (4-6 days), suggesting quick profit-taking opportunities.
Consolidation Characteristics:
Consolidation periods are typically short-lived (2-3 days on average), regardless of overall market conditions.
They often transition to uptrends (42.86% probability), making them potential entry points for long positions.
Volatility Impact:
High volatility shortens the duration of strong trends and increases the frequency of neutral states.
Low volatility extends the duration of uptrends and downtrends, favoring trend-following strategies.
Overbought/Oversold Conditions:
These extreme conditions tend to be short-lived (2-4 days on average), suggesting quick mean reversion opportunities.
Transition Probabilities:
Strong trends have a high probability (>70%) of transitioning to regular trends rather than reversing completely.
Neutral states have a slight bias towards transitioning to uptrends (36.36%) compared to downtrends (31.82%).
Based on some of the principles highlighted we will do the following today:
