How To Write a Book

Or At Least, How I Wrote My Book

We made the Sunday Times bestseller list! Thanks to all of you who pre-ordered the British edition.

Catch-up service:
Podcast: James Marriott on Britain’s Elites
Nine Principles For Success In the Age of AI
What ‘Adolescence Doesn’t Tell Us About Boys’
Did John Lennon Think He Was Jesus?
Why It’s Hard To Stop AIs Lying
The Hipster-Military-Industrial Complex

John & Paul is now published in the UK and US! I’m slightly sad that the era of nagging you to pre-order is over, but it’s very exciting to have it out there, actually being read. I’ve loved hearing what readers have to say about it.

If you are a ‘Founding Member’ paid subscriber, i.e. if you pay £90 or equivalent per year, you can now exercise your right to request a signed copy of J&P from me. Just click on the title of this post, hit reply, and tell me the address you want it sent to, along with any dedication.

If you’re a free or paid subscriber who wishes to upgrade to Founding membership, please be my guest, and make my day. Make sure you’re logged in (https://substack.com/sign-in), go to Settings, scroll down to your Subscriptions, click on The Ruffian and hit the Change button in the Subscription box. You can give a Gift subscription that way too. If you’re a free subscriber, click this green button or the free trial one below.

Paid subscribers make The Ruffian possible, indeed you make all my writing possible, by affording me the time to, er, write.

Quick J&P round-up:

• This week John & Paul was one of eight new books picked as an Editors’ Choice by the New York Times.

• Matthew D’Ancona has written a truly beautiful piece, inspired by the book (and our podcast conversation), about masculinity and The Beatles.

• I was very happy to be a returning guest on one of my favourite podcasts, EconTalk with Russ Roberts, this was a great conversation.

• I talked through some significant J&P songs with Phil Riley on Boom Radio UK.

• This week I’ll be talking J&P in St Andrews, Edinburgh, and Glasgow. Come along if you can!

The Ruffian will shortly revert to normal service, Rattle Bags and all, but this week I have a longer-than-usual piece, for paid subscribers only, on how I came to write John & Paul. I hope it’s both interesting in itself and useful for anyone who has ever considered writing their own book, or indeed embarking on any major creative project.

How To Write a Book (Or At Least, How I Wrote My Book)

John & Paul would not have been possible without The Ruffian. During the lockdowns of 2020, Paul McCartney announced that he had a new album coming out at the end of the year: McCartney III, the third in what would now be a trilogy of experimental solo albums released at the turn of decades (McCartney came out in 1970; McCartney II was released in 1980). I was struck by his sheer creative longevity. McCartney had been writing new songs since 1956. So that was 64 years of making songs, including a song which had made the number 64 famous.

I now had a hook, however contrived, for a piece that had been simmering away in my mind for years: a piece arguing that we still underrate Paul McCartney. It would draw together what I admired about him as a musician, artist and person, and explain why we hadn’t yet seen him straight. I had plenty of notes but now I had some time on my hands, and a framing device. I also had a format: the list. I had a lot to say, and a conventional essay might feel unwieldy or ponderous. I wanted it to snap along like a playlist but to have thematic and narrative coherence.

64 Reasons To Celebrate Paul McCartney turned out to be longer than I’d anticipated: ten thousand words! It could have been longer, however, because I had to cut material about McCartney’s relationship with Lennon. I felt, even then, that the Lennon-McCartney relationship was a whole piece by itself, and maybe even a book.

Having said that, I didn’t think that writing a Beatles book was a realistic prospect for me.

My books had hitherto been about human behaviour and psychology, drawing on the social sciences and the humanities in order to investigate and reframe some aspect of our nature. I had just finished a new book about productive disagreement, which was published in 2021 (postponed from 2020). I wasn’t a music writer and had no credentials in that area. I thought of my essay on McCartney as play rather than work. I didn’t pitch it to a magazine; I was pretty sure nobody wanted to publish a ten thousand word piece arguing that Paul McCartney is good at music. I wasn’t sure anyone wanted to read it, either.

But Substack is the perfect outlet for such follies. If nobody read it, that was fine; at least I wouldn’t have to worry about writing it anymore. So I clicked ‘publish’. Within 24 hours it was clear that it was going insanely viral, mainly by way of Twitter (ah, halcyon days), where it was quickly picked up by some high-profile accounts.

It wasn’t just the scale of the subsequent response that amazed me, it was the emotional force of it. People were moved. They didn’t just like it, they loved it. Some of them cried. I was also struck by the range of different readers. Some were Beatles fans like me, while others were credentialled experts—people who knew everything about the Beatles and had written about them extensively. Yet others—and this, perhaps, delighted me most—were not Beatles fans at all, but found themselves drawn in.

At this point, I started to take the prospect of doing a Beatles book more seriously. I was still a sceptic, but I didn’t have any brilliant ideas for my next book, so in a sense, all my ideas were unlikely. And the McCartney piece had at least partially solved for one of the barriers: I now had some esteemed music writers and musicians saying how much they appreciated my music writing. I had effectively authorised myself to be taken seriously, not least by myself, as a potential author of a Beatles book.

I knew I didn’t want to write a book of the blog post. My first thought was to do a book about the Beatles in the genre I was writing in already – to write a kind of ‘What the Beatles tell us about creativity’ book. But after playing around that with that I realised that all the social science stuff rather paled in comparison to the Beatles stuff. To write about the creation of Strawberry Fields Forever only to then switch to some experiments carried out in a lab risked bathos. The science of creativity just isn’t as interesting as creativity itself.

I realised that if I was going to do this, I’d have to risk doing an entirely new kind of book - for me. A book that told one story and touched only lightly on ideas from the realm of science and psychology. I knew pretty much straight away that if I was going to do this it would be about the Lennon-McCartney relationship. I was amazed and fascinated by their mysterious alchemy and I wanted other people to be amazed and fascinated by it too. My obsession had been fed by a flowering of Beatles podcasts, including ones that focused on the relationships within the group. (One Sweet Dream was particularly stimulating, as was Another Kind of Mind, as was a wondrous YouTube series called Understanding Lennon & McCartney.)

An obvious thing then struck me: there were no Beatles books that focused on Lennon and McCartney. At least, no major books, for a general audience. There were biographies of the group, and there were biographies of individual members. There were hundreds of books that focused on slices of the Beatles story – books about Hamburg or India or the break-up, books about particular albums, or which drugs they took, and so on ad infinitum. But there was no fully-fledged, mainstream study of the group’s central creative partnership. That seemed crazy.

I also came to realise that there were very few Beatles books that I found fully satisfying, as books. I had enjoyed many but, with rare exceptions, the ones that focused on telling the story of what happened were not very insightful about the music, and the ones focused on music were not very good at telling the story. The latter also tended to couch their musical analysis in such technical terms that it was lost on the general reader. And almost none of them delved deeply enough, as far as I was concerned, into the relationships between the band members, from which it seemed to me so much of what’s important about the music flows. Finally, relatively few were well written at the level of prose, or constructed as books rather than as collections of articles.

I didn’t want to write another Beatles book; I wanted to write the best Beatles book. I’m not saying I did so, by the way.¹ I’m just telling you what I was shooting for. I risk sounding arrogant here because I want you to understand the slightly mad conviction I had that my book was necessary. That madness is I think a prerequisite of attempting anything new in a much-ploughed field.

So now I had an idea and at least some confidence that it might work. Except, it wasn’t really an idea. It was a notion.

I was still considering other topics for my next book; topics that were closer to where I’d been before. Around this time I chatted to a friend who is in the book business and is reliably candid about what he thinks will or won’t work. When I floated my thought about Lennon and McCartney, he said something like, “It’s an interesting area, but it’s not an idea yet.” We didn’t discuss it for long but what I took out is that I needed an eye-catching wrapper and format to make this feel like a book waiting to happen.

I knew I wanted to treat the Lennon-McCartney relationship as kind of unconventional love story, but I didn’t have a title. So much comes down to the title and subtitle, which is really the definition of the idea. Around this time (mid-2021) I recorded an episode of the excellent I Am The Eggpod podcast, off the back of my McCartney piece. I talked about McCartney “in twelve songs’. That went well and so I started to think that maybe I could tell the story of the Lennon-McCartney relationship the same way. Over lunch with another friend, a playwright, we came up with “John and Paul: A Love Story in XX songs”. That sounded pretty good.

I also had lunch (sorry about all these chats and lunches, but they were part of the process!) with the historian Tom Holland, an early enthusiast for the McCartney piece. I floated a few half-ideas to him and Tom was very clear that I had to do the J&P book. He told me about a time in his life when he’d been writing vampire novels and decided that he ought instead to write about the Roman Empire, a subject with which he’d been obsessed since he was a kid. He impressed on me that if you have something fresh to say about a topic you love, you should drop everything else and do that. Of course, it helps if the topic you love is also loved by millions of others, but that was true in my case, as it had been in his. The Beatles is not a niche interest.

I now started writing a proposal, but I went about it in an unusual way. I didn’t tell my agent about it, as I normally would. I didn’t mention it to my editor at Faber, whom I liked and trusted. I just put my head down and wrote a proposal for the book that was in my head. Why? Because I was still very doubtful it would work. I didn’t think Faber or any publisher would buy it. There are thousands of Beatles books out there, who needs another one, etc etc. Despite the success of the McCartney piece, I wasn’t a music writer and nor was I a celebrity of any kind. And the idea didn’t fit prevailing trends in publishing. It was about two white men, one dead, the other in his late seventies. It wasn’t exactly discourse dynamite.

In my mind, then, this proposal would fail, but I had to write it in order to not regret not writing it, if you’ll excuse the double negative. I planned to submit it, let it crash and burn, then put it behind me forever and move on. That’s why there was no point in discussing it or in giving extensive thought to where it fitted in the marketplace. My sole aim became to write a proposal for the book I would love to write.

That turned out to be liberating. I hate writing proposals. You spend a lot of time and effort on a document that you know might well turn out to be a complete waste of time. That’s tough. But with this one, I didn’t worry about that prospect, I assumed it. The result is that it was the first proposal I really enjoyed putting together. And the result of that is that it was, surprise surprise, pretty good: it quickly got bought by my publisher here, Faber, and by a first rate publisher in America.

The proposal I wrote was very close to the final book. Apart from one significant detail, it’s the same title and subtitle, the same basic themes, the same structure. This is quite unusual; books often change shape and title during the writing process. Of course, a thousand things occurred to me in the writing of it that weren’t in the proposal, but, structurally speaking, I was very clear in my mind from the start on how I wanted it to work, and what I wanted to say.

The only structural question I hesitated on was whether I should organise the material thematically rather than chronologically. I’d recently read a superb study of Alfred Hitchcock which was organised around themes: his attitude to women, his relationship with his body, his voyeurism, and so on. The reader gets a comprehensive picture of Hitchcock’s life and work, but not in a conventional, one-thing-after-another way. Maybe I could do something like that?

In the end, however, I kept coming back to this: give up on story and you lose emotional traction. Chronological narratives are simply unbeatable at delivering an emotional payload—indeed that might be their primary purpose. Although you can write a fascinating book that isn’t a story, it’s hard to write a truly moving one. And I knew that I wanted this book to move people, because I myself found the Lennon-McCartney story so moving.

Back to that significant detail. The proposal I sold was entitled John & Paul: A Love Story In 23 Songs. It outlined 23 chapters. John and Paul knew each other for 23 years, and this seemed like a good hook to hang the book on. But quite soon after I went to work on the actual book, I realised it was too restrictive. It became clear that I needed to write about more than 23 songs and to highlight more than 23 pivotal moments in their story. It also occurred to me that ’23’ was kind of an arbitrary data point with no real significance. Rather than replacing it with another number, I decided to go with “A Love Story In Songs”, which was actually much better, because it didn’t just describe the book, it described the relationship, as I saw it.

So now I had a simple, direct but evocative title and subtitle, and no restriction on how many chapters I could write. (I ended up with 43 but there’s nothing special about that number; it was driven by the needs of the story rather than any external referent.) I won’t take you through the following three years of actually writing the book in all its gory detail because this piece is already too long (if you have questions, pop them in the comments and I’ll either reply there or write a follow-up). I’ll just a note a few more things about the project.

One is that for a long time after the proposal was bought I still felt quite nervous about the whole thing. I wasn’t planning to do any “primary research” - to turn up blinding new revelations from the archives or to carry out interviews; carrying out new interviews with people who had told their story a hundred times and were now speaking about events of 50 to 70 years ago seemed unlikely to be fruitful. So I’d be relying on the vast amount of material that’s already out there. I was helped by the arrival of Peter Jackson’s Get Back in 2021, effectively a new tranche of primary evidence, not yet incorporated into Beatles literature.

I felt and still feel that was the right way to go about it (as John Higgs put it, we now have plenty of Beatles books about what happened, and a shortage of books about why it happened). It did, however, feel like building a castle in the air. But then again, that’s what any act of creativity, broadly defined, is. You’re building something out of nothing. And in the early stages, while you’re thrashing around in the ether, it can feel very tenuous.

My strategy for coping with this insecurity was to plunge headlong into the first draft; to keep going all the way to the end of the story without looking back – without revising or pondering. I knew that I once I had something to work with, no matter how mediocre or messy, I’d feel more grounded. I’d be able to take a bad book and make it better. It took me about a year to do the first draft and it was the most psychologically punishing, least rewarding part of the process. I spent most of it writing stuff that was, frankly, not very good. As much as I told myself it didn’t matter if it was good or not, that was hard to deal with. It's not fun to end a day’s work without much of anything to look back on with pride.

It was also hard to write from a technical point of view. I’d set myself the challenge of weaving three strands together (without it being obvious that they were strands, if you see what I mean): the narrative (the ‘what happened’), the relationship (what was going on with John and Paul), and the music. I was also trying to appeal to different audiences at once, from Beatles nerds to newbies. So I had to try and write the narrative in a way that didn’t bore the readers who knew it back to front, while equipping those unfamiliar with the Beatles story with what they needed to know (and in the Beatles story, a lot of crucial events happen at pretty much every stage).

Finally, despite not being an ‘ideas’ book, I had plenty of ideas that I wanted to work in, without detracting from the story: ideas about creativity and collaboration and relationships, and of course, ideas about the Beatles and John and Paul. I wanted to weave these ideas in so that they weren’t obtrusive, and to distribute them across the narrative, so that from beginning to end every part of the book resonated with every other part. I tried to follow Gordon Lish’s injunction: “A story must be about what it is about and continue to be about what it is about”.

Inevitably, there were times when I questioned whether my book would turn out to be as good as I wanted it to be, or any good at all. What saved me in those moments was the memory of the reaction to my Substack Beatles posts (both the McCartney one and one I did on Get Back shortly after starting work on the book, which had a similar impact). That was very important in keeping me motivated. Whenever I thought, Why the hell am I doing this, I have no right, this is doomed…I remembered how those posts had gone down. They became the mark I had to hit. All I have to do is write a whole book that’s on the same level as those pieces, I told myself. That seemed possible.

1. For me the gold standard of Beatles books was and remains Revolution In The Head by Ian McDonald - well, that and Can’t Buy Me Love by Jonathan Gould, the book that comes closest to doing all three things I described. In terms of of factual evidence and primary research, Mark Lewisohn’s work is obviously essential too. I highlight a few more in the book’s Notes.