Are we living in a holographic universe?

[abacus9900]

Your effots to explain the calculus showed you to be a fool.

Yeah, I'm still in shock!

But seriously, I have an inkling we're nudging Speaker towards being a tad more 'user-friendly', what do you think?

[Progenitor A]

Jan 24, 2011 21:27:55 GMT 1 abacus9900 said:

Your effots to explain the calculus showed you to be a fool.

Yeah, I'm still in shock!

But seriously, I have an inkling we're nudging Speaker towards being a tad more 'user-friendly', what do you think?

I really think that she is a ranting fool that regurgitates half-digested googles!

[Progenitor A]

Jan 24, 2011 21:15:44 GMT 1 speakertoanimals said:

I have said repeatedly that in a digital encoder the output 00000000 is redundant, contains no information and can be ACTUALLY IS discarded.
In other contexts it may have value as an information string, but if we have an 8-bit repetitive string such as 00000000 00000000 00000000 00000000, in other words if there is no change between the 8-bit strings, then 00000000 00000000 00000000 contain no new information, are redundant and can be discarded. Not only can be, but actually are discarded
But enough of your ignorant half-digested ranting. You are simply a fraud

The only fraud is you, because you are failing to THINK!

The basic shannon result applies (if you read Shannon), to the case where successive bits are independantly and identically distributed. For specific signals (such as a telephone signal) this is not the case.

Lets consider the following. Suppose we have 3 detectors or meters or somesuch, A B and C, whose outputs can be 0 or 1.

We hence have eight possible events, 000, 001, 010, 100, 011, 101, 110, and 111.

IF these are independantly and identically distributed, then the probability for any one is 1/8, which can be encoded in 3 bits (the original thing, of course!).

If we have a signal where 000 occurs a lot of the time, then we have to revise these probabilities accordingly. The more frequent 000 is, the shorter the code word we can use.

Now add a second complication -- if 000 occured 50% of the time but with no temporal correlation, than using -log p would be the best we could do. But suppose instead it occured in chunks of all zeros (no signal in your sense). Then we can do better than the -log p result. in fact, we end up with using end message, start message, and only encoding the other 7 possible configurations.

This is in effect more like your telephony signal, but the difference between the raw information content of 00000000 and in this case is because the real signal doesn't obey the basic constraints in the original shannon result -- that successive states are independantly and identically distributed, with no temporal correlation.

So, we can conclude that correlations, as we might expect, REDUCE the actual information content of the signal. But that doesn't remove the fact that in the iid case with no correlations, the information content of a length N binary message is just N, whether they are all zero, or not.

And I see you have grudingly almost admitted this, with your statement that:

In other contexts it may have value as an information string

You have, I think allowed an engineers bias to enter, and forgotten the basics of information from a mathematical point of view. And I think you have allowed your personal bias against me to stop you considering what I actually said.

So, I return to my original statement. If we were sending a message such as the successive tosses of a fair coin (no temporal correlation), then N is the real information content, even if those N tosses are all zero. Just because for other types of messages with correlations, 000000 may mean something else (the existence of correlations which reduce information content) doesn't negate that result.

Another good example is binary images -- based on the shannon result, we would say information is number of pixels, N. Except if you generate truely random images, they don't look much like actual binary images, because for binary images derived from real images (by processes such as thresholding a grayscale image), there are significant spatial correlations, because real images tend to be of objects, which are blocks of 1's. Hence actual information content can be less than N. Sheer gibberish

Oh go to hell you idiotic waffling buffoon!
I will not spend any firther time with an idiot that cannot explain simple calculus and now shows such enormous ignorance about communication of information

[carnyx]

re STA#104

because for binary images derived from real images (by processes such as thresholding a grayscale image), there are significant spatial correlations, because real images tend to be of objects, which are blocks of 1's. Hence actual information content can be less than N.

So, a row of pixels all containing 1 .. has less information content than
a row with 0s and 1s?

[speakertoanimals]

Oh go to hell you idiotic waffling buffoon!
I will not spend any firther time with an idiot that cannot explain simple calculus and now shows such enormous ignorance about communication of information

So, seems some people are much more interested in scoring silly points and maintaining their own egos in the face of opposition from the very founders of the subject......................

Actually, I feel sorry for you, because if you really do teach, seems you have reached that sad point of being unwilling to keep learning, or unwilling to check your ideas -- seems you need to say -- I know what I know and how dare anyone question it! Except when a clever student brings up the piece from shannon I quoted, what will you do then, teach them, or give them false information in order to protect your own ego?

So, a row of pixels all containing 1 .. has less information content than
a row with 0s and 1s?

Depends, that is the point.

If you are processing real images (ie images generated from real scenes), then there is significant correlation between pixels -- in short, images of real scenes tend to contain areas of similar pixels corresponding to objects and background. Which is why, with limited information, you can do inpainting and approximately reconstruct the image. If you have texture in a scene (such as a woven fabric, say), you can approximately reconstruct a missing piece based on areas around it. Which means that there is less information per pixel in such an image than there is a totally random image.

Where the information theory bit comes in is in the assumption that different pixels are independant. That gives N binary pixels as containing N bits of information. Except that is not the case for real images -- pixels are correlated with others around them, so you can fill them in if they are missing.

Whereas if you had a sequence of coin tosses recorded, if some were missing you could never 'fill them in' -- for a fair coin, knowing that you just threw a head tells you NOTHING about what the next toss was. Hence to get the full sequence of tosses, you need EVERY toss recorded. Whereas if you had a weird coin, where heads tended to occur in runs, or where heads tended to alternate with tails (like woven texture in an image), you could guess some missing ones based on a partial list. This latter case is more like the image case.

So, you can't just say what the information content of a string of pixels IS, without saying a bit more about where they came from. If from a random number generator, then every string contains the same amount of information. But if they come from pictures of coloured squares, say, by knowing part of an image, you can reconstruct the rest. Which is why for real signals, the previous poster got a bit confused, and kept claiming that 000000 contains less information than 010001111 etc.

Where did the extra information GO? Well, one way to think of it is that the extra information that you seem to have lost from the image is in the knowledge you have about what sort of correlations occur in the images you are studying.

To give another example, think of a random sequence of letters -- you can't reconstruct if any of them are missing. You have to know beforehand that what you are working with is a source that generates totally random strings of letters. Whereas if you have a source that generates english words, than you CAN reconstruct some missing letters, based on knowing what all possible english words are. If you have a system that just produces english sentences, then you can have more missing letters and still reconstruct, applying rules of grammar as well as spelling rules. Hence, per letter, a sentence in english contains less information than a random string of letters. Which tends to confuse people, who based on everyday concepts, would see a random string of letters as meaningless (hence how can it have information?) whereas a sentence has meaning, hence must obviously convey more information that a random sequence? You shouldn't confuse information content with meaning.

Enlgihs si atucayll ryev ernuddnta, ouy nac gsseu htis enve hwne lal umjlbde!

[Progenitor A]

Jan 24, 2011 23:38:20 GMT 1 carnyx said:

re STA#104

because for binary images derived from real images (by processes such as thresholding a grayscale image), there are significant spatial correlations, because real images tend to be of objects, which are blocks of 1's. Hence actual information content can be less than N.

So, a row of pixels all containing 1 .. has less information content than
a row with 0s and 1s?

'Real images tend to be of objects, which are blocks of 1's'
Absolute rubbish
Real images do not have 1's or 0's. That is the result of scanning and averaging levels on a threshold detector that covers an area into a binary code
She is absurd.
An empty computer store containing all 0's has as much information content as one filled with changing data- she maintains!
Communication systems do send data when an encoder outputs all 0's -she maintains and proves with a ridiculous logic despite being told of multiple systems that do the opposite of what shesays


Such elementary confusion indicates someone hwo does not undertsnd information transmissio, To mainatin that they are true when all telecommunication systems do the opposite of what she maintains show an incredible obstinate stupidity in the face of the facts

The woman is literally, not meant as an insult, but as an accurate descrition a moron.

[mak2]

It depends what you mean by information.
There is the technical definition.....the number of bits.
Then there is the meaningful information content, if any.

[Progenitor A]

Jan 25, 2011 11:45:04 GMT 1 mak2 said:

It depends what you mean by information.
There is the technical definition.....the number of bits.
Then there is the meaningful information content, if any.

Quite
There is in a digitally transmitted signal, two types of information, user information (sometimes called payload) and control information (sometimes called overhead).
That makes the bit rate higher than is necessary to just transmit user information
However the user information will be processed dependant upon the quality of the transmission channel. In a perfect channel (modern landline transmission channels are evry close to the perfect channel) processing of the user informmation is not necesssary and the overhead wil be simply control bits, such as address bits, frame acknowledgements, compression information, frame numbering etc. The quality of the transmission will depend simply upon the number of bits per sample and the bandwidth that is required for transmission will be just a little more than bit rate/2

For a noisy channel the user information (and control information) must be processed to increase the reliability of the communication channel. You can see the effect of this on your DAB radio channel where the processing of the user information to create a reliable transmision channel considerably delays the transmission of information - you might also notice this on your mobile phone where the transmission channel is particulatly noisy.

Shannon tells us that if we increase the transmision banwidth and add redundancy to the user information then the reliability of the transmission channel is increased. He gives very precise relationships between the channel reliability , the bandwidth of the channel and the type of redundancy coding used.

Error detection and correction bits may also be added, considerably adding to the overhead increasing teh bit rate

The whole business of processing of information to increase realiability is fascinating and without the added redundancy, forward error correction, handshaking, scrambling and inteleaving that goes on in this processing, your mobile phone and DAB radio would be unuseable.
We can thank Shannon that they are useable!

Transmission bit rates of up to 10x the user bit rate are quite commonplace

[speakertoanimals]

'Real images tend to be of objects, which are blocks of 1's'
Absolute rubbish
Real images do not have 1's or 0's. That is the result of scanning and averaging levels on a threshold detector that covers an area into a binary code
She is absurd.

In an earlier post, I did specifically refer to thresholding grayscale images. Even without that, what I meant is clear from the context.

An empty computer store containing all 0's has as much information content as one filled with changing data- she maintains!

I don't, Shannon says it does! Or it does until you further specificy that we are not dealing with all possible configurations of that data storage system, but that particular sub-set that we come across in certain computer systems.

WHY you think static information isn't information is beyiond me, and sheer stupidity -- a written page is INFORMATION, and it doesn't change.

I refer you back to SHannon and the first page of the paper I quoted.

Such elementary confusion indicates someone hwo does not undertsnd information transmissio, To mainatin that they are true when all telecommunication systems do the opposite of what she maintains show an incredible obstinate stupidity in the face of the facts

Its become quiite obvious that YOU are the one who doesn't understand either Shannon 9log measure of information content), or how you go from his basic results to the results for real transmission systems. That is what you are missing, the connection between what you think you understand as an engineer, and the basics of information theory.

I have explained where your confusion probably comes from, I have given various examples, yet you are still stuck in this daft position. If even Shannon can't convince you (I note you never attempted to discuss the direct quotation), then I have to dismiss you as that sad type of teacher who really is clueless when it comes to the basis of their subject, even if they can apply it on a practical level.

The woman is literally, not meant as an insult, but as an accurate descrition a moron.

The only moron here is you mate!

Seems to me you have grudingly admitted I am right, but still maintain this smokescreen that I'm not, because in practical applications, strings of repeated bits are the rule (and mean nowt), rather than the exception. Which is why these real-world messages don't correspond to the strings Shannon started by considering, there are significant temporal correlations, which is WHY you can do bloody data compression, in effect. You seem to be unable to grasp this, and instead have some bastardised half-understanding of shannon that you transmit to your poor bloody students.

It depends what you mean by information.
There is the technical definition.....the number of bits.
Then there is the meaningful information content, if any.

Let me try a few more quotes:

In information theory, the term information is used in a special sense; it is a measure of the freedom of choice with which a message is selected from the set of all possible messages. Information is thus distinct from meaning, since it is entirely possible for a string of nonsense words and a meaningful sentence to be equivalent with respect to information content.

Hence a register all zeros is probably not very meaningful, but does contain as much information! note the phrase 'set of all possible messages' -- I SAID I was specifically considering all possible strings of N bits, NOT strings of bits likely to come out of digitizing a phone call. A list of coin tosses is one example, where HHHH contains EXACTLY the same amount of information as HTTH (or any other 4 toss string).

[speakertoanimals]

That is the result of scanning and averaging levels on a threshold detector that covers an area into a binary code

I should just say, this is WRONG. You threshold by looking for pixels greater than some value, whereas averaging doesn't give you a binary image, but just a smoothed grayscale image.

The information theory stuff does confuse, we just think that 000000 intuitively contains less information. And some developments in algorithmic complexity incorporate this.

What is going on here is about the source. SO, IF I have a fair coin, a string of 100 successive heads does have as much information content as any other specific string of 100 tosses.

BUT if instead I can asking about whether or not the coin is fair, and I toss it 100 times, and get 100 heads, then comparing the probability of getting 100 heads from a fair coin (small), with the probability of getting at least one head (almost 1), compared to the probability of getting 100 heads from an unfair coin, I could legitimately conclude that my coin WASN'T fair, and that in that case, 100 heads contains no information! Whereas if I had a more random string of mixed heads and tails, I'm more likely to conclude I have a fair coin. If however when I tossed it again, I got the SAME sequence of heads and tails, then I might conclude that it wasn't a fair coin at all!

So then, for a given message, perhaps we need to do two things -- ask what that message possibly tells us about the process generating it (fair coin or not), AND then ask what the information content is given those assumptions about the coin.Which gets us closer to Rissanen and the Minimum Description Length.

But if we already know that the coin is fair, and heads and tails really are random, then any string contains just as much information as any other, which is the result Shannon used for the logarithmic measure of information content -- N bits is N bits.

And if we know that we are dealing with a signal-generating process where there are significant temporal correlations (such as phone signals with periods of quiet), then the actual information content will be less than the Shannon measure (hence lossless compression has a chance of working).

[speakertoanimals]

Re Transmission post -- except we WEREN'T talking about transmission (where adding redundancy increases reliability for noisy channels -- this is, I contend fairly bleedin' obvious!)

BUT:

How much information is actually in a signal?

Given that, can we compress it or not, and what is the best way to encode that for transmission.

Which is all the -log (p) stuff -- which then gives information content of a string of zeros as the same as any other bit string of the same length -- under the assumptions that we have applied (iid bits).

When you've sorted all that stuff out and what to send a whole bunch of such messages down a noisy communication channel, that is a slightly different problem. You may understand the latter, doesn't mean you actually understand the mathematical basis of the former.

Your friendly neighbourhood moron...................

[abacus9900]

Jan 25, 2011 15:22:00 GMT 1 speakertoanimals said:

That is the result of scanning and averaging levels on a threshold detector that covers an area into a binary code

I should just say, this is WRONG. You threshold by looking for pixels greater than some value, whereas averaging doesn't give you a binary image, but just a smoothed grayscale image.

The information theory stuff does confuse, we just think that 000000 intuitively contains less information. And some developments in algorithmic complexity incorporate this.

What is going on here is about the source. SO, IF I have a fair coin, a string of 100 successive heads does have as much information content as any other specific string of 100 tosses.

BUT if instead I can asking about whether or not the coin is fair, and I toss it 100 times, and get 100 heads, then comparing the probability of getting 100 heads from a fair coin (small), with the probability of getting at least one head (almost 1), compared to the probability of getting 100 heads from an unfair coin, I could legitimately conclude that my coin WASN'T fair, and that in that case, 100 heads contains no information! Whereas if I had a more random string of mixed heads and tails, I'm more likely to conclude I have a fair coin. If however when I tossed it again, I got the SAME sequence of heads and tails, then I might conclude that it wasn't a fair coin at all!

So then, for a given message, perhaps we need to do two things -- ask what that message possibly tells us about the process generating it (fair coin or not), AND then ask what the information content is given those assumptions about the coin.Which gets us closer to Rissanen and the Minimum Description Length.

But if we already know that the coin is fair, and heads and tails really are random, then any string contains just as much information as any other, which is the result Shannon used for the logarithmic measure of information content -- N bits is N bits.

And if we know that we are dealing with a signal-generating process where there are significant temporal correlations (such as phone signals with periods of quiet), then the actual information content will be less than the Shannon measure (hence lossless compression has a chance of working).