Copyright is authorized by Article I section 8 of the Constitution:
The Congress shall have Power... To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries;Technically, one would expect the First Amendment to override any conflicting parts of the original Constitution, just as the Eleventh and Twenty-First Amendments did. However, when James Madison proposed the Bill of Rights, he said it was almost completely unnecessary and would change almost nothing. Moreover, the first American copyright law was passed by the First Congress one year after the Bill of Rights. Therefore, the First Amendment and the Copyright clause cannot contradict each other.
The book is not the author's property; otherwise, copyright would last forever. Rather, as the Supreme Court said in Harper and Row v. Nation Enterprises, "copyright supplies the economic incentive to create and disseminate ideas... 'the ultimate aim is, by this incentive, to stimulate [the creation of useful works] for the general public good.'" In other words, while the First Amendment prevents government from impeding "uninhibited, robust, and wide-open" debate, copyright stimulates it by giving authors an incentive to write. If it restricts your right to freely speak the author's exact words, it "does not impede First Amendment goals because the public purpose has been served - the public already has access to the idea or the concepts." You can simply say "Read N.'s book - I agree with every word of it!" (If you don't, you can still make fair use of it for "criticism" or "comment.")
However, what if your friends don't have access to the book? In 2004, Presidential candidate John Kerry said he supported the Vietnam War. However, James Davis posted online a 1971 book Kerry had written against the war. To protect his image, Kerry had Davis's website taken down as a copyright infringement. Even outside of politics, the long terms of copyright have left many works unavailable. Many copyright holders been lost or refused permission for republication! Currently, there is no recourse except (sometimes) a long lawsuit.
Courts have said this does not matter, since you are free to use the ideas found in the book, essay, or story. For example, Suntrust v. Houghton Mifflin allowed Alice Randall to publish a parody of Gone with the Wind, holding that it made transformative use of Michell's work. However, this is not enough. If Davis had just published a summary of Kerry's book, voters would have dismissed it as unfairly biased. Without the original text (and with Kerry probably publishing a competing summary), who could decide? Fortunately for Davis, the First Amendment protects not just ideas but also expressions. If the State of California must allow Cohen to say "[obscenity] the Draft" when he could have expressed his opposition to it in many other ways, then "governmental bodies may not prescribe the form or content of individual expression." If a particular expression (copyrighted work) is not available, the government has no business keeping people from making it available. Copyright is supposed "to promote the Progress of Science and useful Arts" and fill the marketplace of ideas; why should we allow authors to use it to empty that marketplace and retard the progress of knowledge?
Indeed, Suntrust hints that any use for "purposes such as criticism, comment, news reporting, teaching . . ., scholarship, or research" is fair use, saying that thanks to the First Amendment, "any use of a copyright is permitted to fulfill one of the important purposes listed in the statute." Clearly, the republication of Kerry's book could be called "news reporting," and republication of orphaned works could be called "scholarship." Unfortunately, courts do not apply the fair use doctrine in this manner. For example, Nihon Keizai Shimbun, Inc. v. Comline Bus. Data, Inc., after finding that the defendant excerpted the plaintiff's articles "for the purpose of news reporting", denied fair use protection on the grounds that "Comline's infringing abstracts are 'not in the least 'transformative.''"
If we let people republish works not available, it would not strip any profits from the author; if the work is not available, the author is not getting any profits in the first place. Indeed, the Republican Party would probably have been quite happy to pay Kerry royalties for reprinting his book. Similarly, as Google is scanning orphaned works, it is putting money away to pay copyright holders when they are located. The principle beneficiary, however, is the public: "the ultimate aim is, by this incentive, to stimulate [the creation of useful works] for the general public good."