Newsgroups - Advanced Topics

Propagation

Just about every ISP in the world has a news server to which its customers read, and post messages, and upload and download files. The news servers of the world are linked to each other and automatically exchange messages; a process called “propagation.” So, when you connect to your ISP’s news server and look in alt.fan.howard-stern, you see all the messages posted by everybody on the planet. This is much different than web discussion boards, which are fragmented into zillions of tiny boards. You would have to pay a visit to each web site to see everything everybody is saying.

When you post a message, it will usually propagate around the world within seconds. But if your ISP is having a technical problem, your post may only show up on your news server. There is no way to tell if this has happened just by looking at your news server. If nobody mentions your post, then it may not have propagated. The only way to tell for sure is to look on a different news server.

Expiration and Retention

The posts in a newsgroup are numbered as they arrive - either via propagation from other servers, or from being posted directly by customers. So, the first post that arrives is number 1. The next is 2, and so on into the hundreds of millions. News servers cannot keep everything posted forever as it would cost a fortune to do so, so old posts are “expired” on a regular basis, which simply means that they are removed. Typically, posts are numbered something like: 123,456 through 987,654, etc. The period in which posts are kept is called the “retention” period. So, one news server may have a retention of 30 days, and another may have a retention of 45 days. In general, the longer the better.

Completion

“Completion” is what percentage of large files the server has. So, if a DVD is posted in 1,000 parts, and the server has 990 of them, then its completion is 99%. Servers will typically be missing small numbers of posts because of imperfect propagation. If the file is posted by somebody in Japan, it must travel through a few different news servers, and networks, before it gets to your ISP’s server. Various glitches along the way can cause a post to get lost. To deal with this problem, most large files are posted with PAR files.

Encoding

As we discussed, files must be encoded into text. There is no official standard for encoding, and you will see posts with files encoded in yEnc, UU, and BASE-64. You don’t need to worry about encoding too much as your news-reader program will handle it automatically. The only thing to be aware of is that Outlook Express is not able to decode yEnc files.

News Service Providers (NSP’s)

There is an entire industry of companies that operate news servers and sell access to them. We are an NSP, and anybody in the world can sign up for an account and access our server. NSP’s exist because many ISP's offer lack-luster service. For example, AOL and MSN use outdated technology that prevents their customers from using just about all newsgroup software. Their customers have two choices: 1) cancel their account and sign up with a normal ISP, or 2) keep their account and sign up for a news account with an NSP, which can be accessed through any internet connection.

Many ISP’s also “throttle” their news servers to deliberately make them run slower. This saves them money on network costs. They may sell you a very fast internet connection, but their news server will crawl along. In general, the free news server that your ISP provides is fine for use with discussion groups. But if you want to download lots of files at high speed, an account with an NSP is usually needed.

Continue on with: Newsgroup Software