What is an index?
Introduction
A stock market index was originally invented in 1896 by Charles Dow as a way to measure the performance of the stock market. Before the age of computers, it was difficult to know how well the stock market as a whole was performing. It was too difficult back then to add up the values of all stocks being traded, so Mr. Dow judgmentally picked a handful of stocks that he thought would be representative of the stock market as a whole, thus inventing the Dow Jones Industrial Average. The idea was that by tracking on a daily basis the ups and downs of the key stocks in the Dow Jones Industrial Average Index, an investor would have a good gauge of how well the stock market as a whole was performing. The original Dow Jones Industrial Average literally added up the trading prices of the 12 stocks in the index and divided that number by 12 (since then, the Dow Jones Industrial Average has been expanded to 30 stocks).
Standard & Poor's launched the S&P 500 Index in 1957. The S&P 500 Index consists of 500 large companies that Standard & Poor's decided were representative of the stock market as a whole. Although most of the companies in the S&P 500 Index are large companies, the S&P 500 Index is not necessarily the largest 500 companies on the stock market (there are a few large companies not included in the S&P 500 Index). There is a committee at Standard & Poor's that periodically adjusts the 500 stocks in the index so that the S&P 500 Index continues to be "representative" of the stock market.
The S&P 500 Index was the first "market capitalization" index. Rather than merely adding up the trading prices of the stocks in the Dow Jones Industrial Average, the S&P 500 adds up the market capitalization of each stock in the index. Market capitalization is merely the trading price per share multiplied times the number of shares of common stock. So if McDonald's stock is trading at $10 per share and McDonald's has 1,000,000,000 shares outstanding, McDonald's has a market capitalization is $10 billion. With a market capitalization index, each stock carries a different weight in the index. If the 500 stocks in the S&P 500 Index have a total market capitalization of $100 billion, and McDonald's has a market capitalization of $10 billion, then McDonald's has a "weight" in the S&P 500 Index of 10%.
The big 3 indexes
Because of their historical importance, the Dow Jones Industrial Average, which is often simply referred to as "the Dow", and the S&P 500 Index are still widely followed. Virtually every news source that publishes something on the stock market mentions how the Dow and the S&P 500 have been performing. But they also usually mention a third index: the NASDAQ-100 Index, which is often referred to simply as "the NASDAQ". The Dow, the S&P 500 and the NASDAQ have become a "big 3" of indexes that are mentioned and discussed everywhere.
To a large degree, the Dow and the S&P 500 are similar indexes: they are both a hand picked selection of stocks that are intended to be representative of the stock market as a whole. But the NASDAQ is quite a bit different. The NASDAQ is made up of the 100 largest companies that are listed on the NASDAQ stock exchange that are not considered to operate in the financial sector of the economy. As explained in our article about U.S. stock exchanges, only about 40% of all stocks are listed on the NASDAQ stock exchange, and the New York Stock Exchange ("NYSE") has a bigger market share than NASDAQ. So why would such a "narrow" index become so important? Because the NASDAQ stock exchange has done a good job of attracting growth stocks during the past thirty years, the NASDAQ-100 Index has become a great "technology" index: Apple, Microsoft, Amazon, Google and Facebook are all listed on the NASDAQ and included in the NASDAQ-100 Index. You don't have to be a technology company to be included in the NASDAQ-100 Index (Pepsi and Costco are both members) but through either good luck, or perhaps by design, the NASDAQ-100 Index has become one of the best indexes to use to track the technology revolution that has happened during the past thirty years.
The Russell Indexes
In addition to the Big 3 indexes, every investor should also understand three Russell family indexes that are widely followed:
- The Russell 1000 Index consists of the 1,000 largest U.S. common stocks (including REITs) based on market capitalization. Remember, the S&P 500 Index is not made up of the largest 500 U.S. common stocks, but rather 500 stocks picked by a committee.
- The Russell 3000 Index consists of the 3,000 largest U.S. common stocks (including REITs) based on market capitalization. Although there are usually more than 3,000 U.S. common stocks on the stock market, the Russell 3000 Index is a widely followed index that tracks "the total market".
- The Russell 2000 Index consists of the 2,000 smallest U.S. common stocks (including REITs) based on market capitalization that are included in the Russell 3000 Index. So the Russell 2000 Index is a widely followed "small cap" index.
The growth of index investing
The original purpose of an index was to measure the daily progress of the stock market. But stock market indexes took on a whole new meaning in the early 1970's when investment funds were launched that intentionally bought all the stocks in the S&P 500 Index, rather trying to individually pick and choose stocks to purchase. A mutual fund that bought and sold stocks based on an index like the S&P 500 Index was called an "index fund".
Index funds steadily grew in popularity from 1970 to the early 2000s, as more and more investors decided that it was easier, and less expensive, to invest in an index fund. During this time period, more and more research came out that documented how difficult it is for a portfolio manager of an investment fund to "outperform the market". In fact, most mutual funds run by a portfolio manager actually had poorer results than most index funds, partly because most mutual funds run by a portfolio manager included fairly high annual fees that cut into the results of the fund. Index funds, on the other hand, tended to have much lower annual fees.
The debate between the merits of "active investing" versus "index investing" continues to rage on, but over the past twenty years investors increasingly have flocked towards index investing. When exchange traded funds, or "ETFs", were invented in 1993, it was the perfect marriage of a new type of investment fund that had certain structural advantages over a mutual fund, and the growing trend towards index investing. So ETFs became synonymous with "index investing", even though not all ETFs track an index. And ETFs became very popular.
Index symbols
We should note that indexes have ticker symbols, just like stocks. But there is no simple, clear system of giving every index a symbol that is universally used. The index provider refers to the index using one symbol, and then the two major professional data services (Bloomberg and Reuters) will use their own index symbols. And finance websites like Google Finance and Yahoo Finance will use another symbol.
Let's look at an example. S&P states on their website that the S&P Composite 1500 Index has a ticker symbol of SPR. If you download the Factsheet from S&P's website, it also lists a Reuters ticker symbol of .SPSUP. On Google Finance, the symbol "SPR" doesn't work but you can pull up the index using both SP1500 and SPSUPX. On Yahoo Finance, you can pull it up using the symbol ^SP1500. On the Wall Street Journal's website, you use SPSUPX. Its very confusing. We don't know where the symbol "SPR" used by S&P actually works.
Index types
There are three different types of indexes:
- Price index
- Total return index
- Net total return index
A price index tracks the market prices of the securities in the index, and ignores the value of any dividends on the underlying stocks or interest on the underlying bonds. The S&P 500 Index and the Dow Jones Industrial Average are price indexes.
A total return index is the price index itself plus the value of any dividends on the underlying stocks or interest on the underlying bonds. A net total return index is the price index plus the value of any dividends on the underlying stocks or interest on the underlying bonds, net of some theoretical tax on those dividends and interest.
Net total return indexes are primarily used in the context of international stocks and bonds. International taxation is complex, because every country has a different way of taxing the income of foreigners who buy and sell stocks and bonds in their country. Many countries require companies to withhold tax from any dividend or interest payments made to a foreign investor. So there are quite a few net total return indexes that calculate the net return to a U.S. investor, net of these automatically required withholding taxes.
Note that commodity indexes use different terminology, as explained in our article understanding commodity ETFs.
Index weighting
As explained above, when the S&P 500 was launched in 1957, that started a trend towards indexes weighting their holdings based on each stock's market capitalization. Indexes were originally designed to measure how well the stock market, or a portion of the stock market, was performing, so weighting the holdings based on each stock's market capitalization made sense.
As the trend towards "index investing" became more popular, someone came up with the idea of building an index that didn't just track the performance of the market, but actually tried to outperform the market. Can you build an an index that systematically uses a set of rules to pick stocks, in a way that will outperform the market, rather than just tracking the market? Or can you build an index that might outperform the stock market as a whole by weighting the stocks in the index using a method other than using market capitalization? This line of thinking created an explosion in the index industry, spawning thousands of new indexes.
In the early 2000's, someone coined the term "smart beta" to refer to an ETF that was tracking one of these new style indexes that was built to outperform the market. With the explosion in popularity of smart beta ETFs since then, all kinds of indexes have been launched that weight their holdings in different ways. By weighting their holdings in a different way, these smart beta indexes hope to outperform the market, or achieve some other desired benefit such as a higher dividend yield or lower volatility.