Talk:Markov chain

Awful article

Scientific articles are Wikipedia's Achilles heel, but this is truly terrible. I'm trying to understand section A non-Markov example and I'm giving up. That's just not the way to do it. The counter-example should be simple and clear, not so complicated and obscure. And if you do such complicated examples, you need to add drawings.

It is also questionable description in Applications. Statistics? It is all statistics. Economics and finance? Ok, but next Social sciences includes economics. Finance could be separated from economics and then social sciences combined with economics. Pawel.jamiolkowski (talk) 18:32, 9 June 2024 (UTC)

Hi @Pawel.jamiolkowski,

For your feedback to be useful, you have to be more specific. Is your problem with the stochastic process itself, or with the way it is presented? How do you think that a drawing could help here?

Regarding your comment on the section Applications: listing "statistics" is perfectly relevant; please have a look into the difference between probability theory and statistics to see why the study of Markov chains is not a subfield of statistics. I think grouping economics and finance is also relevant here, but I agree that something more specific than social science could be used (especially considering that the meaning of the word is a bit vague and has changed over time). However to me these are pretty minor issues.

Malparti (talk) 08:31, 10 June 2024 (UTC)

First of all, A non-Markov example is the sub-section of Examples, right? Reader reads examples which are brief description without details. And it's okay. And then we asking: Ok, so what process won't be Markov chain? The reply is below. However instead seeing a similarly short description, we see loooong piece of writing.

It should be consistent with the previous part. Short decription for intuitive understanding.

I don't know what drawings could be included here, because I don't understand this.

And secondly, why is a separate article with the same title? I guess to deepen the topic. There should actually be such a counter-example there (although better written anyway - first an intuitive description, then a deeper explanation). Pawel.jamiolkowski (talk) 13:49, 10 June 2024 (UTC)

Hi again @Pawel.jamiolkowski,

The subsection is actually rather short; what makes it look long is in part the explanation of how to build a Markov chain out of this non-Markovian process (which is somewhat irrelevant — here we want to understand what makes a process non-Markovian; not how to make non-Markovian processes Markovian by augmenting the state-space...). My guess is that this is a recent addition; articles on basic and widely used math tools have a natural tendency to grow because they attract a lot of readers from different backgrounds and many people want to add something — so they have to be trimmed down once in a while. I will probably remove the last paragraph of the subsection, but there are other problems I would like to fix first.

Another problem might be the use of the terms "dime" and "nickel" and "quarter", which lengthens the text and make is look more complex than it is to non-American readers. I guess that could be improved, too.

Other than that, I think this example is about as simple at it can get without getting something entirely trivial, and I think the level of detail is more-or-less OK. So I think it would be helpful if you could invest time to understand it and then explain what you found hard to understand, and how the description could be improved.

Best, Malparti (talk) 22:27, 10 June 2024 (UTC)

You have almost the same example, but simple and clear here: Markov property

Do you see? It should look like that. 3 balls, colors etc.

But generally for grasping idea, at the beginning it should just explain that the essence of difference lies in difference between draw with replacement and draw without it. It's obvious that if we have constant number of objects and draw without replacement, the future depends on draw, because each draw corresponds with each ball / coin, 1:1. As the number of objects decreases, it is obvious that the probability of guessing increases. Everyone can understand it without any problem. Pawel.jamiolkowski (talk) 23:47, 10 June 2024 (UTC)

I agree that this example has some advantages; but I disagree that it is strictly better, at least in the way it is presented. For instance, it is not explicitly stated what is the process of interest. This is not ideal, especially since if we were to model everything that is described verbally, we would of course get a Markovian process. But I agree that something like "the color of the n-th draw in a Polya urn process" seems like a more natural counter-example than the example described in the article. I will try to take care of this over the weekend. Malparti (talk) 13:32, 12 June 2024 (UTC)

Unnecessary citations

Just dropping in to say that a book being part of ChatGPT['s] training set [1] is the worst argument for citing a reference that I have seen in weeks.

Also, the opening line of the "Applications" section is a summary of the text that follows and doesn't need a citation itself. XOR'easter (talk) 20:47, 12 June 2024 (UTC)

@XOR'easter: Indeed... So it seems my concerns were not unfounded. ;). It also seems some people know how to use different IPs.

If that's OK with you I've moved your comment to a new section, to keep it separated from the discussion with Pawel.jamiolkowski.

Best, Malparti (talk) 23:05, 12 June 2024 (UTC)

Semi-protected edit request on 13 June 2024

HI,the page was bloked, however, some users abused the system and erased Gagniucs book with no reason. I wish to insert Gagniucs book again, and the main reason is the fact that is the most academically cited book listed on this page. More ... the suspicious reason for wich a top book was erased on no grounds at all except replacement with other unknown sources that have no academic grounds to be cited. 213.233.108.161 (talk) 20:40, 13 June 2024 (UTC)

Declined. There was no "abuse" of "the system". Gagniuc's book was removed for multiple reasons, any one of which would have been sufficient by itself. "Most academically cited book listed on this page" is a meaningless standard, particularly since the book could well have benefited from having been advertised here for so long. XOR'easter (talk) 20:51, 13 June 2024 (UTC)

Gagniuc's book coasted on the free publicity of being prominently cited on Wikipedia for a long time, but the edit-warring to keep it there brought it to the attention of people whose idea of a good time is editing mathematics articles on Wikipedia. The result: despite the book being called "from theory to implementation", there is not an ounce of theory — most of the basic concepts are not presented. The book is full of approximations, and the way things are written gives the impression that the author does not understand the basics... Not to mention the >100 pages of computer code which I doubt anyone is ever going to read (it is even hard for me to comprehend how Wiley could agree to print something like this in 2017). And likewise: 60 pages of badly-worded boring worked examples with no theory before we even get to the possibility of having more than two states. As Malparti said, there is no theory, or rather theory is alluded to in vague and inaccurate form without any justification. For instance the steady state (still of a two-state chain) is first mentioned on 46 as "the unique solution" to an equilibrium equation, and is stated to be "eventually achieved", with no discussion of exceptional cases where the solution is not unique or not reached in the limit, and no discussion of the fact that it is never actually achieved, only found in the limit. Do not use for anything. I should have taken the fact that I could not find a review even on MR and zbl as a warning. Fuller discussion can be found at the WikiProject Mathematics talk page (permalink). XOR'easter (talk) 18:18, 30 June 2024 (UTC)

Problems with Ergodicity section.

Firstly, there is a paragraph "If all states in an irreducible Markov chain are ergodic, then the chain is said to be ergodic. Equivalently, there exists some integer $k$ such that all entries of $M^{k}$ are positive."

My question is: what is $M$?

Previously in the article the letter $M$ is used only for mean hitting time $M_{i}$ of state $i$. I suspect it was meant to use stochastic matrix $\mathbf {P} ^{k}$.

Secondly, the sentence "More generally, a Markov chain is ergodic if there is a number N such that any state can be reached from any other state in any number of steps less or equal to a number N." seems wrong. The part "any number of steps less or equal to N" implies that all states can be reached in 1 step (as "1" is "any number less than N"). This is not true for all the ergodic Markov chains. Removing final "any" should fix the statement.

Still, the statement doesn't align with the previously given definition of ergodicity as it doesn't exclude Markov chains with periodic components. 2001:BB8:2002:98:7987:3E47:DD3E:2E0A (talk) 09:03, 13 September 2024 (UTC)

Proposal to reintroduce peer-reviewed source (Wiley, 2017)

Hello everyone,

I would like to propose reintroducing a citation to a peer-reviewed academic book published by Wiley in 2017:

📘 Markov Chains: From Theory to Implementation and Experimentation by P.A. Gagniuc.

ISBN: 978-1-119-38755-8

The book provides a clear, structured overview of both the formal definitions and the historical development of Markov chains, including references to the work of Andrey Markov himself.

The source is:

• Peer-reviewed and published by Wiley (a major academic publisher)
• Cited over 1,000 times in the academic literature
• Directly relevant to the introductory definitions in this article

I understand concerns about citations in the lead, but in this case, the source is used to support factual information about definitions and history, not any personal opinion or promotional claim.

Suggested reinsertion:

The theory of Markov chains, which are mathematical systems that transition from one state to another within a finite or countable number of possible states, has its foundations in the early 20th century through the work of Andrey Markov. A comprehensive overview of both the historical context and formal definitions is provided by Gagniuc (2017), in a peer-reviewed academic volume published by Wiley.<ref>Gagniuc, P.A. (2017). Markov Chains: From Theory to Implementation and Experimentation. Wiley. ISBN 978-1-119-38755-8.</ref>

Please let me know if there are any policy-based objections to this. Otherwise, I plan to reinsert it into the article in 48 hours. EricoLivingstone (talk) 12:16, 31 March 2025 (UTC)

Hi @EricoLivingstone.

Please see the following discussion: Advice on dealing with questionable citations in lead. I also encourage you to have a look at the book itself.

I'll summarize the main points here. First, concerning the book itself:

• it is full of inaccuracies (and even blatant mistakes) on the basics;
• It is far from being a comprehensive overview of the subject: it covers only a small subset of what most textbooks on Markov chains cover;
• This last point is subjective, but all of the other editors who have had a look at the book seemed to agree with me: the book is "outstandingly poorly" written (please have a look at the 100+ pages of printed computer code and tell me how this can useful to anyone).

Since there is no shortage of excellent textbooks of Markov chains, there is no good justification to cite Gagniuc's book rather than one of the classics. Also, the fact that it was published by Wiley doesn't give it any right to be cited on Wikipedia: it merely makes it eligible for that. Not every published book / scientific article has to be on Wikipedia.

Second, there has been some shady business going on around Gagniuc's book. In particular, its introduction on Wikipedia seems to have resulted from a concerted spamming effort by both registered editors (e.g, the now-banned User:MegGutman) and various IP users (mostly from Romania). It is likely that this has contributed to the 1000+ citations that you mention. As a fun exercise, you can have a look at the articles that cite Gagniuc, and how they cite it.

I agree that I should have linked to the original discussion in my comment (even though, as the admin who closed your request pointed out, it wasn't too hard to find). I apologize for not doing it.

Best, Malparti (talk) 15:02, 31 March 2025 (UTC)

Thank you, Malparti. I respectfully disagree with several of your points, and I’d like to respond with policy-based reasoning:

1. WP:RS and WP:SCHOLARSHIP – The book is peer-reviewed, published by Wiley (a recognized academic publisher), and cited in over 1,000 scholarly works. That meets Wikipedia’s standards for a reliable academic source.
2. WP:NPOV – Your concerns about “poor writing” or “mistakes” are subjective and not sufficient for exclusion. Content inclusion is not based on personal taste but on verifiability and reliability. Please cite specific factual errors with page numbers if that’s your concern.
3. WP:OR – Suggesting that the book gained citations through “shady business” or “spamming” is speculative and outside the scope of content policy. Unless there’s a formal ruling by admins or the RS noticeboard, such claims are considered original research or conspiracy.
4. Relevance – The book provides a historical and definitional overview, which justifies its use in the lead or relevant sections. It’s not the only source, nor is it replacing the classics. Including multiple perspectives is encouraged under WP:DUE.

I appreciate you clarifying your views, but unless specific factual inaccuracies are documented with references, I will proceed to reinsert the citation as proposed. EricoLivingstone (talk) 15:38, 31 March 2025 (UTC)

I agree with Malparti. The consensus of multiple editors in the linked discussion on Wikipedia talk:WikiProject Mathematics/Archive/2024/Jun#Advice on dealing with questionable citations in lead is that this is a low-quality source for this topic that does not stand up to scrutiny. We have better. There is no policy that states that we must use a source merely because it has been published; we invariably have a choice for which source to use, and should make that choice based on our judgement of the quality of the source. Your insistance on spamming this source does not appear to be grounded in any encyclopedic purpose and raises suspicion that you are related somehow to the blocked editors previously blocked for promotional edits of the same material. —David Eppstein (talk) 17:37, 31 March 2025 (UTC)

Thank you for your note, David. However, I respectfully disagree with your characterization.

• There is no formal consensus against this source — only informal agreement among a small group of editors who also happen to be actively opposing the citation across multiple pages.
• Wikipedia policy (WP:RS and WP:SCHOLARSHIP) does not require a source to be the “best” or to win a subjective popularity contest. If it is published by a reputable academic press and cited widely, it qualifies as reliable.
• Suspicion of connections to other editors is speculative and violates WP:AGF (Assume Good Faith). I’d prefer to keep this discussion focused on content policy and verifiability, rather than speculative assumptions about editor identity.

Unless a clear policy-based reason is presented — with page-specific counter-arguments — I will proceed with the reinsertion, in accordance with WP:V and WP:DUE. EricoLivingstone (talk) 18:08, 31 March 2025 (UTC)

Google Scholar search for "Markov chain" finds some 2 million hits, most presumably reliably published. Are you trying to suggest that policy requires us to list them all as 2 million footnotes in this article? If not, what makes you think this one source has any justification for special treatment, different from the 1999999 others?

As for suspicion of connections: You could explicitly state that you have neither a commercial nor personal connection to Gagniuc nor to User:MegGutman, rather than merely deflecting the issue. —David Eppstein (talk) 18:18, 31 March 2025 (UTC)

Thank you, David.

I never claimed that a source’s inclusion should be based solely on citation count — only that the combination of publisher reliability (Wiley) and academic citation (1,000+) meets the threshold under WP:RS and WP:SCHOLARSHIP.

Regarding your second point: I'm not suggesting we list all sources found via Google Scholar. But when a source is peer-reviewed, relevant, and cited in academic literature, its inclusion should be discussed on those merits, not dismissed by analogy.

As for the request to "explicitly state" my personal affiliations — I respectfully decline. Wikipedia policies do not require editors to disclose personal or professional identities. Disputes should be resolved through content policy, not speculation.

Let’s focus on verifiability, reliability, and relevance as outlined in WP:V, WP:RS, and WP:NPOV. EricoLivingstone (talk) 18:29, 31 March 2025 (UTC)

Please note that I have not asked you to disclose your identity. I have asked you to disclose your conflicts of interest, if you have them, as is required by Wikipedia's terms of use. THe fact that you keep refusing to respond constructively to such a request is not reassuring. —David Eppstein (talk) 20:35, 31 March 2025 (UTC)

I invite you to continue adding the material against the clear consensus of all competent editors who have considered the question, but only because that is the most direct route to your being blocked for disruptive editing. --JBL (talk) 17:57, 31 March 2025 (UTC)

Also note forum-shopping at WP:AN#Improper removal of Wiley academic source from "Markov chain" article — possible coordinated abuse and WP:DRN#Markov chain. —David Eppstein (talk) 18:06, 31 March 2025 (UTC)

Thank you, JBL. I’m aware of Wikipedia's conduct guidelines, and I’m committed to editing in accordance with WP:V, WP:RS, WP:NPOV, and WP:DUE.

I note that your comment does not address the actual content of the source or cite any policy-based reason for exclusion. Warnings about “being blocked” are not a substitute for verifiability or discussion based on Wikipedia’s content standards.

If you believe my editing is disruptive, the appropriate venue is WP:ANI. Otherwise, let’s please keep this focused on policy, not intimidation. EricoLivingstone (talk) 18:24, 31 March 2025 (UTC)

• Just to note that this has been brought up on RSN. I suggest the discussion continue here rather than splitting it. -- LCU ActivelyDisinterested «@» °∆t° 18:26, 31 March 2025 (UTC)

  Thank you, @ActivelyDisinterested. I appreciate your note.

  I agree that centralizing discussion is helpful. I raised the issue on RSN and DRN only after Talk page attempts failed and the source was repeatedly removed with non-policy-based reasoning.

  I’m happy to follow consensus-building wherever volunteers believe is most appropriate, as long as it remains focused on policy (e.g., WP:RS, WP:SCHOLARSHIP, WP:DUE).

  I appreciate your input and willingness to help guide the process. EricoLivingstone (talk) 18:51, 31 March 2025 (UTC)

  It would be better to follow consensus-building, period. You are not the arbiter of whether other people's thoughts are valid or not - it is not up to you to judge what is acceptable reasoning and what isn't. You should be looking for compromise - for example, by finding some other citation that can support whatever the text you want to add might be. MrOllie (talk) 19:02, 31 March 2025 (UTC)

  Thank you, MrOllie. I appreciate your input.

  I’d like to clarify that I’m not acting as an “arbiter” of anyone’s reasoning — I’m simply requesting that content decisions be based on Wikipedia’s established standards (WP:RS, WP:SCHOLARSHIP, WP:V), not on subjective impressions or editor preferences.

  The Wiley source (Gagniuc, 2017) is peer-reviewed, published by a respected academic publisher, and cited in over 1,000 scholarly works. This meets all criteria for inclusion as a reliable source.

  I welcome compromise when multiple valid sources exist, but replacing a compliant source with subjective dismissals is not aligned with WP:DUE or WP:NPOV.

  If there are policy-based objections with specific examples (e.g., factual errors or RS policy violation), I’m open to discussing them. Otherwise, I believe the source deserves inclusion. EricoLivingstone (talk) 19:12, 31 March 2025 (UTC)

  Reliability doesn't guarantee inclusion. Whether to include content is an WP:NPOV matter, any included content must be verifiable to a reliable source. As long as content has a source there is no need to include more sources, Wikipedia isn't a list of all sources for a subject. The citation you added was redundant, so there is no policy based reason to reintroduce it - regardless of whether or not it is reliable. I'll leave the reliability of mathematical works to WikiProject Mathematics, who appear to have already discussed it. -- LCU ActivelyDisinterested «@» °∆t° 20:04, 31 March 2025 (UTC)

• Noting that the instigator of this thread has been blocked indefinitely. --JBL (talk) 21:31, 31 March 2025 (UTC)

A decent source

• Vogt, W. Paul (2005). "Markov Chain". Dictionary of Statistics & Methodology: A Nontechnical Guide for the Social Sciences. SAGE. p. 186. ISBN 9780761988557.

A good rule of thumb for citations of introductory definitions is a good glossary or a "Dictionary of X" book. They are usually pitched at the right level, as this one would seem to be from its title. ☺ It pretty much entirely supports the first two sentences. This would leave no excuse at all for that other source to keep cropping up. Vogt taught statistics and was a professor at Illinois State University, according to xyr 2016 obituary. Uncle G (talk) 22:02, 31 March 2025 (UTC)