Claude Elwood Shannon

Characters

ClaudeElwoodShannon(ClaudeElwoodShannon,1916-2001)wasbornonApril30,1916inPetoskey,Michigan,USA.HegrewupinthesmalltownofGaylord,whichhadonlythreethousandinhabitantsatthetime.Thefatheristhejudgeofthetown,andtheirfatherandsonhaveexactlythesamename,andtheyarebothClaudeElwoodShannon.HismotheristheprincipalofthemiddleschoolinthetownandhernameisMabelWolfShannon.Hegrewupinawell-educatedenvironment,butthescientificinfluencehisparentsgavehimdoesnotseemtobeasgreatastheinfluenceofhisgrandfather.Shannon'sgrandfatherwasafarmerandinventor.Heinventedwashingmachinesandmanyagriculturalmachinery,whichhadadirectimpactonShannon.Inaddition,Shannon'sfamilyhasadistantrelationshipwiththegreatinventorEdison(ThomasAlvaEdison,1847-1931).ShannonspentmostofhistimeatBellLabsandMIT(MassachusettsInstituteofTechnology).After"achievementandfame",ShannonandMary(MaryElizabethMoore)marriedonMarch27,1949.TheymetatBellLabs.Marywasadataanalystatthetime.Theyhavefourchildrenintotal:threesonsRobert,James,AndrewMooreandonedaughterMargaritaCatherine.Laterthereweretwolovelygranddaughtersbesidehim.

OnFebruary24,2001,ShannonpassedawayinMedford,Massachusetts,attheageof84.BoththeobituariesissuedbyBellLabsandMIThonorShannonasthefounderoftheeraofinformationtheoryanddigitalcommunications.

Character'slife

ShannonwasborninPetoskey,Michigan,USAonApril30,1916,andisadistantrelativeofEdison.GraduatedfromtheUniversityofMichiganin1936withabachelor'sdegreeinmathematicsandelectricalengineering.In1940,hereceivedadoctorateinmathematicsandamaster'sdegreeinelectricalengineeringfromtheMassachusettsInstituteofTechnology(MIT).HejoinedtheMathematicsDepartmentofBellLabsin1941andworkeduntil1972.HebecameavisitingprofessorattheMassachusettsInstituteofTechnology(MIT)in1956,alifetimeprofessorin1958,andanhonoraryprofessorin1978.Dr.ShannonpassedawayonFebruary24,2001,attheageof84.

ShannonbeganthinkingaboutinformationtheoryandeffectivecommunicationsystemsduringTheInstituteforAdvancedStudyatPrincetonin1940.After8yearsofhardwork,Shannonserializedandpublishedafar-reachingpaper"TheMathematicalPrinciplesofCommunication"intheBellSystemTechnicalJournalinJuneandOctober1948.In1949,Shannonpublishedanotherfamouspaper"CommunicationunderNoise"inthemagazine.Inthesetwopapers,Shannonclarifiedthebasicproblemsofcommunication,gaveacommunicationsystemmodel,proposedmathematicalexpressionsfortheamountofinformation,andsolvedchannelcapacity,sourcestatisticalcharacteristics,sourcecoding,channelcoding,etc.Aseriesofbasictechnicalissues.Thetwopapersbecamethefoundationalworksofinformationtheory.

Shannonreceivedabachelor'sdegreeinmathematicsandelectricalengineeringfromtheUniversityofMichiganin1936,andthenenteredMITtostudyasagraduatestudent.

In1938,Shannonreceivedamaster'sdegreeinelectricalengineeringfromMIT.Thetitleofhismaster'sthesiswas"ASymbolicAnalysisofRelayandSwitchingCircuits".Atthattime,hehadnoticedthesimilaritybetweentelephoneswitchingcircuitsandBooleanalgebra,thatis,the"true"and"false"ofBooleanalgebraandthe"on"and"off"ofthecircuitsystemwerecorrespondedandrepresentedby1and0.SoheusedBooleanalgebratoanalyzeandoptimizeswitchingcircuits,whichlaidthetheoreticalfoundationfordigitalcircuits.ProfessorHowardGardnerofHarvardUniversitysaid,"Thisisprobablythemostimportantandfamousmaster'sthesisofthiscentury."

In1940,ShannonreceivedhisPh.D.inmathematicsatMIT,buthisdoctoralthesiswasRegardinghumangenetics,thetitleis"AnAlgebraforTheoreticalGenetics"(thealgebraoftheoreticalgenetics).ThisshowsthatShannon'sscientificinterestsareverywide,andhelaterpublishedmanyinfluentialarticlesindifferentdisciplines.

Whilestudyingforhisdegree,healsospentpartofhistimestudyingdifferentialanalyzerswithProfessorVannevarBush.Thisanalyzerisanearlymechanicalsimulationcomputerusedtoobtainnumericalsolutionsofordinarydifferentialequations.Shannonpublished"Mathematicaltheoryofthedifferentialanalyzer"in1941.Hewrote:"Mostresultsaregivenintheformofproventheorems.Themostimportantthingistodealwithsomeconditions,someconditionscanbeGenerateafunctionofoneormorevariables,someconditionscanmaketheordinarydifferentialequationbesolved.Italsogivessomeprecautions,givingtheapproximatevalueofthefunction(cannotproduceanaccuratevalue),theapproximatevalueoftheadjustmentrateandtheautomaticcontrolrateMethod."

In1941,ShannonenteredtheAT&TBellTelephoneCompanyinNewJerseyasamathematicsresearcher,andworkedatBellLabsuntil1972,from24to55yearsold,for31years.HebecameavisitingprofessoratMITin1956,becameafullprofessorin1958,andretiredin1978.

PeopledescribeShannon’slife.Duringthedayhealwaysclosedhisdoortowork,androdehisunicycletoBellLabsatnight.HiscolleagueD.Slepianwrote:“Weallcometoworkwithlunchandplaymathgamesontheblackboardafterdinner,butClauderarelycomes.Healwaysworksbehindcloseddoors.ButifyouTofindhim,hewillhelpyoupatiently.Hecanimmediatelygrasptheessenceoftheproblem.Heisreallyagenius.AmongthepeopleIknow,Ionlyusethiswordforhim."

ShannonworkedwithJohnRiordanandpublishedapaperonthenumberofdualterminalsinseries-parallelnetworksin1942.ThispaperextendsthetheoryofthepaperpublishedbyPercyA.MacMahon(1854-1929)onElectricianin1892.

In1948,theepoch-making"amathematicaltheoryofcommunication"wasdividedintotwopartsandpublishedintheBellSystemTechnicalJournalinJulyandOctober.Thearticlesystematicallydiscussesthedefinitionofinformation,howtoquantifyinformation,andhowtobetterencodeinformation.Inthesestudies,probabilitytheoryisanimportanttoolusedbyShannon.Shannonalsoproposedtheconceptofinformationentropytomeasuretheuncertaintyofthemessage.

Inthelongyears,hethoughtaboutmanyissues.ExceptforoneyearofworkingatthePrincetonInstituteforAdvancedStudy,hemainlyspenthistimeatMITandBellLab.ItshouldbenotedthatduringWorldWarII,Dr.Shannonwasalsoawell-knowncodebreaker(thisremindspeopleofDr.Turing,whois4yearsolderthanhim).HisdecipheringteamatBellLabismainlytotrackGermanaircraftandrockets,especiallywhentheGermanrocketsplayedabigroleintheblitzkriegagainsttheUnitedKingdom.In1949,Shannonpublishedanotherimportantpaper"CommunicationTheoryofSecrecySystems".Itisbasedonthisworkpractice,anditsmeaningistomakeconfidentialcommunicationfromarttoscience.

In1948,Shannonpublished"AMathematicalTheoryofCommunication"inBellSystemTechnicalJournal.Thepaperwasco-signedbyShannonandWeaver.WarrenWeaver(1894-1978)wasthedirectoroftheNaturalSciencesDepartmentoftheRockefellerFoundationatthetime,andhewrotetheforewordforthearticle.Later,Shannonstillengagedintechnicalwork,whileWeaverstudiedthephilosophicalissuesofinformationtheory.Bytheway,whenthepaperwasfirstpublished,theindefinitearticleAwasused,andwhenitwasincludedinthecollectionofpapers,itwaschangedtothedefinitearticleThe.

Achievementandhonor

Achievement

Theconceptofentropy

ImportantfeaturesofShannon’stheoryItistheconceptofentropy.Heprovedthatentropyisequivalenttothedegreeofuncertaintyofinformationcontent.EntropywasonceaconceptintroducedbyBoltzmanninthesecondlawofthermodynamics.Wecanunderstanditasthechaosofmolecularmotion.Shannonextendedtheconceptofentropyinstatisticalphysicstotheprocessofchannelcommunication,thuscreatingthesubjectof"informationtheory".The"entropy"definedbyShannonisalsocalled"Shannonentropy"or"informationentropy",whereimarksallpossiblesamplesintheprobabilityspace,indicatingtheprobabilityofoccurrenceofthesample,andKisanarbitraryconstantrelatedtounitselection.Itcanbeclearlyseenthatthedefinitionof"informationentropy"andthedefinitionof"thermodynamicentropy"(Boltzmann'sformula)differonlybyacertainconstantofproportionality.

Entropy(entropy)referstothedegreeofchaosinthesystem.Ithasimportantapplicationsincybernetics,probabilitytheory,numbertheory,astrophysics,lifesciencesandotherfields.Itisalsoextendedindifferentdisciplines.Themorespecificdefinitionofisaveryimportantparameterinvariousfields.EntropywasproposedbyRudolfClausiusandusedinthermodynamics.Later,ClaudeElwoodShannon(ClaudeElwoodShannon)firstintroducedtheconceptofentropyintoinformationtheory.

Asweallknow,quality,energyandinformationarethreeveryimportantquantities.

Peoplehaveknownforalongtimetousescalesorbalancestomeasurethemassofmatter,andtherelationshipbetweenheatandworkwasnotuntilthemiddleofthe19thcentury,withtheclearnessofthethermalpowerequivalentandtheestablishmentofthelawofconservationofenergy..Thetermenergyistheirgeneralterm,andthemeasurementofenergyissolvedbytheemergenceofnewunitssuchas"caloriesandjoules".

However,theknowledgeofwords,numbers,pictures,andsoundshasahistoryofthousandsofyears.Butwhatistheirgeneralterm,howtomeasurethemuniformly,untiltheendofthe19thcenturyhasnotbeencorrectlyputforward,letalonehowtosolveit.Atthebeginningofthe20thcentury,withthedevelopmentoftelegraph,telephone,photo,television,radio,radar,etc.,thequestionofhowtomeasuretheamountofinformationinasignalwasvaguelyputontheagenda.

In1928,R.V.H.HarleyconsideredtheproblemoftakingoutNsymbolsfromDdifferentsymbolsandforminga"word".Iftheprobabilityofeachsymbolisthesame,anditisselectedcompletelyatrandom,D^Ndifferentwordscanbeobtained.TakingaspecificonefromthesewordscorrespondstoaninformationvolumeI.HartleysuggestedusingNlogDtorepresenttheamountofinformation,thatis,I=NlogD.Thelogheremeansthelogarithmtothebase10.Later,Wiener,thefounderofcyberneticsin1949,alsostudiedtheproblemofmeasurementinformationandledittothesecondlawofthermodynamics.

ButShannonisthecorefigurewhogavethebasicmathematicalmodelforinformationtransmission.In1948,Shannon'stensofpagesofthesis"TheMathematicalTheoryofCommunication"becameamilestoneintheofficialbirthofinformationtheory.Inhiscommunicationmathematicalmodel,theproblemofinformationmeasurementisclearlyraised.HeextendedHartley'sformulatothecasewheretheprobabilitypiisdifferent,andobtainedthefamousformulaforcalculatingtheinformationentropyH:

Ifthelogarithminthecalculationisbasedon2,thenthecalculatedinformationentropyisinbits.WordssuchasByte,KB,MB,andGB,whicharewidelyusedincomputersandcommunications,allevolvedfrombits.Theemergenceof"bits"signifiesthathumansknowhowtomeasuretheamountofinformation.Shannon'sinformationtheorymadeadecisivecontributiontoclarifyingtheconceptofinformationcontent.

WhenShannonperformedquantitativecalculationsofinformation,heclearlydefinedtheamountofinformationasadecreaseinthedegreeofrandomuncertainty.Thisshowshisunderstandingofinformation:informationisusedtoreducerandomuncertainty.OrShannon'sinversedefinition:Informationisanincreaseincertainty.

AlthoughShannon’sconceptofinformationhasmadegreatprogresscomparedwithpreviousunderstandings,itstillhaslimitations.Thisconceptalsodoesnotincludethecontentandvalueofinformation.Itonlyconsiderstheuncertaintyofrandomness.Answerthequestionof"whatisinformation"fundamentally.

Infact,Shannon’soriginalmotivationwastogetridofthenoiseinthephone.Hegavetheupperlimitofthecommunicationrate.Thisconclusionwasfirstusedonthephone,thentheopticalfiberwasused,andasof2013,itwasusedinthewirelessCommunication.Ourabilitytomakeoverseascallsorsatellitecallsclearlyiscloselyrelatedtotheimprovementofcommunicationchannelquality.

ClaudeShannonisnotparticularlywell-knownamongthepublic,butheisoneofthefewscientistsandthinkerswhoenableinstantcommunicationinourworld.HeisamemberoftheAmericanAcademyofSciences,amemberoftheAmericanAcademyofEngineering,amemberoftheRoyalSociety,andamemberoftheAmericanPhilosophicalSociety.Hehaswonmanyhonorsandawards.Forexample,theMorrisAwardin1949,theBallantineAwardin1955,theKellyAwardin1962,theNationalMedalofSciencein1966,theHonorMedalofIEEE,theJaquardAwardin1978,theFritzAwardin1983,andtheKyotoAwardforBasicSciencein1985.Thehonorarydegreesheacceptedaretoonumeroustolist,soIwon’trepeatthem.

WemissShannon,andwemustbefamiliarwithhistwomajorcontributions:oneistheconceptofinformationtheoryandinformationentropy;theotherissymboliclogicandswitchtheory.Weshouldlearnfromhiscurious,practical,andnever-satisfiedscientificspirit.Thisisanimportantexperienceforhissuccess.

Honors

AlfredNobleSocietyAmericanEngineerAward1940

MorrisLiebmannRadioEngineersSocietyMemorialMedal1949

YaleUniversity(ChiefScientist)1954

StuartBallantineFranklinSocietyMedal1955

ResearchCooperationAward1956

UniversityofMichigan,HonoraryDoctorate1961

RiceUniversityHonoraryMedal1962

PrincetonUniversityHonoraryDoctorate1962

MarvinJ.KellyAward1962

EdinburghHonoraryDoctorate1964

UniversityofPittsburghHonoraryDoctorate1964

InstituteofElectricalandElectronicEngineersHonoraryMedal1966

TheNationalMedalofSciencewasawardedbyformerPresidentLyndonB.Johnsonin1966

GoldenPlateAward1967

NorthwesternUniversityHonoraryDoctorate1970

HarveyPrize,theTechnionofHaifaIsrael1972

OxfordUniversityHonoraryDoctorate1978

JosephJacquardAward1978

HaroldPenderAward1978

UniversityofEastEnglandHonoraryDoctorate1982

CarnegieMellonUniversityHonoraryDoctorate1984

AmericanAudioTechnologyAssociationGoldMedal1985

KyotoPrize1985

TuftsUniversityHonoraryDoctorate1987

UniversityofPennsylvaniaHonoraryDoctorate1991

EduardRheinPrize1991

CharacterMemorial

• ShannonAward

TheShannonAwardisanawardsetuptocommemoratetheAmericanfounderofinformationtheory,ClaudeElwoodShannon.Itisthehighestawardinthefieldofcommunicationtheoryandisalsoknownasthe"NobelPrizeinthefieldofinformation."