In video terminology container or container format means a "wrapper" format, which doesn't define what compression formats (in other words, what codecs) the video file within a specific container format includes, but rather defines how the video, audio and other possible data is stored within the container.

Explaination

A compression format is a algorithm used to compress audio/video data. MP3 is a popular audio compression format. Simmiliarly, DivX, MPEG-4 etc are common video compression formats. Codec (abbreivation of "Coder/Decoder"), is a program that understands compression format, and is capable of performing transformations on data stream. Basically it is a piece of software or a driver that adds a support for certain video/audio format for your operating system.

Many multimedia data streams need to contain both audio and video data, and often some form of metadata that permits synchronization of the audio and video. Each of these three pieces of data may be handled by different programs, processes, or hardware; but for the multimedia data stream to be useful in stored or transmitted form, they must be encapsulated together. Container is way of "wrapping" audio, video streams with metadata in a single file.

AVI: AVI (Audio Video Interleaved) is one of the most commonly used container formats. It was developed by Microsoft in early 90's.
ASF: Advanced Streaming Format (or ASF, later renamed into Advanced Systems Format) is Microsoft's proprietary digital audio/digital video wrapper, especially meant for streaming.
MOV/Quicktime: QuickTime is a digital video technology developed and produced by Apple Computer. In 1998, the ISO approved the QuickTime file format as the basis of the MPEG-4 standard, with supporters noting that it was a good "life-cycle" format, well-suited to capture, editing, archiving, distribution, and playback (as opposed to the simple file-as-stream approach of MPEG-1 and MPEG-2, which are poorly suited to editing).
Real Media: This was developed by Real Networks as media streaming solution.
Note: Open source implementations of mpeg program streams, avi and quicktime containers also exists.

Ogg

Ogg Media Container is developed by Xiph as the framework of a larger initiative aimed at developing a set of components for the coding and decoding of multimedia content which are both freely available and freely re-implementable in software. The Ogg bitstream is defined in RFC 3533 and its MIME media type (application/ogg) in RFC 3534. This format was designed with main aim of media streaming (but is good for all common needs). More information about ogg can be found here (which is probably outdated).

To start using Ogg, just read Ogg howto.

OGM

Tobias created DirectShow filters for the Ogg Multimedia Format which can be used to encode video into the Ogg format, as well as play Ogg Vorbis files on programs like Windows Media Player. The problem is that the DirectShow filters are an extension to the Ogg Multimedia Framework that lies outside of the Ogg specification. So, while it is certainly useful, it's not really 'Ogg.' As it is important to maintain control of the specification in these matters Xiph does not support OGM. OGM and Ogg are not same.

Please read more about OGM to know more about OGMs.

MCF

Meida Container Format (MCF) is yet another open source project aiming to become the standard of Multimedia Container Formats one day. But MCF is more or less a dead project now.

Matroska

Matroska is derived from MCF and is aiming to compete with Quicktime and MOV (and replace AVI) as a general use container with a clear focus on video and audio editing. More information about matroska can be obtained from http://www.matroska.org.

Ogg and Matroska have different philosophies. Ogg is focussing on high compatibility and playability, making it ideal for content distribution, as well as streaming. matroska on the other hand is competing with MOV and Quicktime, and aiming to replace AVI and VfW/Dshow, as a general use container ( kitchen sink format ) with a clear focus on video and audio editing. Ogg has much lower and bounded overhead, and its primary design distinction is streamability, while Matroska offers hierachical access, very much in the vein of Apple's Quicktime file format. Read more about Ogg Vs Matroska.

Ogg format have id header which are an additional restriction as compared to OGM. So while an OGM file is an Ogg file an Ogg file is not necessarily an OGM file, and so parsers that assume the id header will break on general ogg files (anything but degenerate Ogg Vorbis basically). And of course, we believe it to be an unnecessary abstraction breaking restriction designed to make the format easier for AVI-based code to parse. Since the whole point was to get around the limitations of AVI for content distribution, it irks, and that's why we don't support it.

Also people generally use the patent encumbered DivX or mpeg4 video codecs in OGM with vorbis audio codec. This gives a false sense of open source solution (becase DivX/mpeg4 are not open).

Read more about OGM.

You can of course use whatever extension you like, but the recommended extension is .ogg. RFC 3534 defines application/ogg mimetype to represnt Ogg files.

Please checkout Ogg Writ.

As of now for most common usages, we recommend you to use
Ogg Vorbis for audio compression and
Ogg Theora for video compression.
Other compression formats are flac (audio compression), speex (speech data compression) and Tarkin (experimental video compression).

Ogg is the umbrella for a group of several related multimedia and signal processing projects that are open source and royalty free. Development of these projects is controlled by Xiph.org. For more information go to Ogg project page.
Ogg files can be easily used on both Linux and Windows. Just read Ogg howto and start using Ogg.