War and Peace Text Analytics

IT 546 Data Mining · Text Analytics · Walsh College · Fall 2024

War and Peace Text Analytics

Her Excel workflow (tokenize, count, chart the top words, score sentiment with a word dictionary) re-run live in JavaScript on the complete text of Tolstoy's novel.

Text miningTokenizationWord frequencyLexicon sentimentExcel (COUNTIF)JavaScript
…words tokenized
…distinct words
…chapters
…of dictionary hits are positive

Live demo

The full novel is decompressed and tokenized in your browser when the page opens. Every control below recounts from those tokens.

Word frequency

Find any word

Compare characters

Counts exact word matches per part, possessives included ("Pierre's" counts as pierre). "Andrew" is how this translation names Prince Andrei.

Results

Dictionary sentiment, as in her Excel line chart: count positive and negative lexicon words in each section and plot the net score in reading order.

Net sentiment per chapter (positive minus negative words, per 1,000 words)

Chapter scoreMoving average

Positive and negative word rates by part

Positive words per 1,000Negative words per 1,000

How it works

  1. Clean. Start from the Project Gutenberg plain text with the license header and footer removed, then detect the 15 books, 2 epilogues and every CHAPTER heading so each word knows where it sits.
  2. Tokenize. Lowercase the text and split it into words on anything that is not a letter, the scripted version of her Text to Columns step. Possessive "'s" is dropped so "Pierre's" counts as pierre.
  3. Count. Tally every word, the equivalent of her COUNTIF column, then filter by stop words and minimum length and sort for the top-N bar chart.
  4. Score sentiment. Match each word against a simple dictionary, like her Excel lookup list: 150 positive and 150 negative words written for this page. Net score = (positive minus negative) per 1,000 words.
  5. Chart the trend. Plot net sentiment chapter by chapter in reading order and smooth it with a centered moving average, as her line chart did across sections.

Limits of a word-list method: it ignores negation ("not happy" scores positive), sarcasm and context, and the lexicon is small. "War" and "peace" are left out of the dictionary so the title words do not drive the score. Treat the curve as a rough tone indicator, not a reading of the book.

Show the dictionary

Positive:

Negative:

Data

War and Peace by Leo Tolstoy, translated by Louise and Aylmer Maude, Project Gutenberg eBook #2600.

Chapter table (first 10 of 365 rows)

The text ships inside this page gzip-compressed (about 1.2 MB instead of 3.2 MB) and is unpacked by the browser's built-in DecompressionStream.