How to Retire at Age 55 – An Interview with Fritz Gilbert of The Retirement Manifesto

Not long ago, a fellow retirement blogger retired at the age of 55. I have the same retirement goal! So when I read about his triumphant exit from corporate America, I thought, who better to interview about how to retire at age 55 than Fritz Gilbert of The Retirement Manifesto?

Fritz spent 30+ years at one company, building a successful career by working his way up the corporate ladder. But every time his income increased, he kept his lifestyle and spending habits the same.

This enabled him to save diligently over the years, eventually becoming a 401(k) millionaire, and then some. But the 401(k), he says, was not the key to his early retirement. 

Now that he’s retired, he spends his time on landscaping projects, writing, traveling and helping his wife start a charity that helps dogs find a different kind of freedom (check out the video at the end of the interview). Next, he’s headed off on a great American road trip in a camper to visit his baby granddaughter for the summer. 

We first met at a bloggers conference in 2017, before he had all this time. Last week, I spoke with Fritz on the phone for about an hour and recorded our conversation.

I used this opportunity to pick his brain about how to retire at age 55. We talked a lot about retirement cash management tactics, but also career growth, the FIRE movement, staying busy in retirement, and the secret to early retirement (sorry, that part is super-boring).

This is the first time I’ve interviewed anyone on my website. I hope you enjoy it. Let me know what you think in the comments. 

Thanks to Fritz for joining me for this conversation!

RBD: You retired at age 55… Was that always that goal? When did you realize 55 would be the time to leave work?

Fritz: I was atypical of the FIRE mentality. But I was always saving aggressively. When our income went up, our lifestyle didn’t. We took any pay increase and saved it.

I always thought in the back of my mind, I’d like to retire early. But we weren’t obsessed with a date. We did pay for our daughter’s college education, then we kept saving to see where things would go. I wasn’t focused as much on the goal, as much as just enjoying the moment. We took vacations all over the world.

Then in my mid 40’s, I had some friends that were retired early. So I started thinking about how well we were doing and thought, when do you think we could do this? In my late 40’s, we started putting numbers to it.  

By 50, I was tuned into the idea of retiring at 54. Then I decided to work one more year and retire at age 55. So we never had a long-term goal like you, it was more – let’s just save and see how early we can get out.

RBD: Was your friend who retired early a corporate friend?

Fritz: Yeah, a guy that I worked with was a really good friend of mine. He was closer to the FIRE mentality, he probably saved 60%-70% of his salary, and he retired at about 45.

Then he bounced back and forth, getting more opportunities for work. He’d take a year or two off, then get a very profitable gig for a year, then leave again. 

When I saw him do this, we were about the same age, and I said, you know what, I think we can do this too. And that’s when we got serious about setting a date.

RBD: Part of the reason I got serious about dividend investing was to bridge the gap between age 55 and 59 1/2 when we can access retirement money without penalty. How do you intend to bridge that gap to pay for living expenses today and until age 59 1/2?

Fritz: Well actually, if you work until you’re 55, you can access your 401(k) without penalty. A lot of people don’t know that. So I can access my 401(K), and that was part of my logic of working one more year. I was thinking about 54, but if I worked for one more year, I wouldn’t have to worry about bridging at all. 

The reality is we built up enough after-tax money to be able to bridge. We don’t expect we’ll have to access the 401(k). And we’re actually withdrawing a lot less than we originally thought we would. So we built in sufficient after-tax money and my pension – I was fortunate to get a corporate pension. So between the pension and after-tax, it’s enough to get us to 59 1/2.

RBD: Is that the 72(t) rule that allows you to withdraw money from your 401(k) without penalty at age 55?

Fritz: Yes.

And really for me, there were a couple of points about working one more year. First, I had an uncle who retired early. I saw him at a wedding, and he had one piece of advice for me. He said that if you’re pretty sure you’re ready to retire, but not really sure, put in an extra year because you’ll never make the money you’re making now. 

Second, a friend of mine who retired about two years before me, and he was 55, I told him about my uncle’s advice. He followed that advice and worked the extra year. After his first year of retirement, he said Fritz, that was the best piece of advice you gave me. Because now we know we’ve got enough. 

Third, knowing in a pinch we could access the 401(k) if we needed it. We didn’t expect to, but it’s a nice-to-have. 

RBD: Are you selling after-tax investments for living expenses, or living off of dividends, or have you created income streams to fund your lifestyle?

Fritz: Well, I originally changed our dividends so they were flowing into our money market account. But I undid that because we’re pulling so little cash out. 

The thing that I wasn’t planning on, is about two weeks before I retired, a friend in the industry called me with a tempting offer. 

If you read the post I wrote called, When at Work, Work!, I wrote about a huge presentation I gave at a conference a few years back. I worked my butt off for this presentation, and I crushed it. 

Well this guy happened to be in the audience. He called me later on. I thought he was calling to wish me well in retirement. But this guy’s company had recently restructured, and they were putting together a new Board of Directors, and he asked if I would like to be on the board.

And I was like, now that is interesting! So I talked to my wife about it, and we thought it would be a nice way to stay engaged in the industry. 

I did not have to do anything for pay, so that wasn’t much of a factor. The thing was, I was two weeks away and really wanted the freedom of retirement. 

It turns out, they are very flexible and being on the Board is not invasive at all. It takes a few hours a month with a very little bit of travel every year. With the compensation, combined with my pension and a little bit of blog income, we’re pulling much less out of savings than we anticipated. 

RBD: Funny how that happens. You put all this thought and time into thinking about how you’re going to bridge that gap, and then a good thing happens. And you know, that’s actually an argument for retiring earlier. People often think they need more money than they actually do. 

Fritz: Yeah, and that’s sometimes a criticism of FIRE, people say you’re not actually retired because I’m working on a board. Well, yes, I am retired. I don’t work. I exercise every morning, I’m helping my wife start a charity. Trust me, I know what it’s like to work and this is not work! It’s nothing like work. 

The reality is when you’re driven enough to retire early, you’re probably pretty good, and you’re going to get opportunities. And the beauty of it is, I didn’t choose it for the financial side, I’m doing it for the other benefits, and it just happens to have the money that comes with it. 

RBD: You worked for a large corporation 30+ years, and things seemed to work out OK, right? But do you ever feel like staying with the same company for so many years prevented you from reaching your real potential? Or were there more opportunities in your company since you stayed for so long?

Fritz: When I was hired out of college, I did the interview as a practice interview. I had no interest in the aluminum industry. And I started my first job and I started to kind of like it. Nice culture. I liked the people.

Then within a year, I got my first promotion. Within another two years, I got another promotion. Every two-to-three years I was moving cities, and I was changing jobs and the people I was working with. So the job stayed fresh. That happened for most of my career. 

I went from being a sales guy to being a plant manager, to being a part of the North American management team, to being part of the global management team. That is really why I stayed because it wasn’t boring. It was giving me new challenges. 

Along with that, as you get into your career, you start thinking about the pension. The first 15 years, I didn’t care. But I started thinking if I can ride this job out and I’m enjoying it, there’s a huge pay off with the pension.

I did have a job opportunity elsewhere about five years into my career. But I used that as leverage within my company and got a raise out of it.

Being within the company, I knew the people, I knew I was positioned well, I was connected to the right people, and things were moving favorably in my career. 

If I would leave, especially if the next job was not significantly better than the one I had, I always knew I’d be at that next level soon anyways. So I was always able to get promoted without leaving the business.

RBD: I will never have a pension! But my new company has an incredible retirement plan and is a great place to work, so I’m hoping to stick it out.

Fritz: There were a lot of lifers at my company which is a good sign. When you’re with a good company, there’s a lot more to it than jumping ship for a 10% raise. There’s more to it than the money side of it. If you’ve got it good, hey, be thankful, because most companies don’t have good cultures, and culture is a big part of work.

RBD: Alright, let’s talk about your retirement withdrawal strategy. In what order are you drawing down your assets? I know you’ve said you’d leave the Roth IRA untouched for as long as possible. Tell me more about the Roth.

Fritz: I’ve written a lot about this. In my working years, I executed the mega-backdoor Roth conversion a few times. And I’m planning to covert more before-tax money to Roth over the next few years. I’ll pay taxes on it now, but I’d rather get it out now than hold it in pre-tax accounts (IRA/401(k)s).

As for my safe withdrawal rate, I was originally targeting something like 3.25% as a maximum. 4% is too aggressive for me at current valuations.

It turns out that after working that one extra year plus earning some extra income, we’re below out 3.25% safe withdrawal rate. So it’s working out better than expected.

Also, terms of sequence and withdrawal, I’m using the bucket strategy:

• Bucket #1 has liquid assets
• Bucket # 2 has income producing assets 
• Bucket # 3 is for long-term growth

Some people think it’s too conservative because you have too much in cash. But personally, we don’t need the extra returns because our withdrawal rate is so conservative.

I like having a couple of years of liquidity so we can ride out a bear market. We have a couple of years worth of liquidity sitting in liquid assets such as a high yield savings account, a Vanguard money market, and short-term bonds. 

As we draw those down, we start selling stuff out of buckets two and three, long-term bonds, stocks, alternative investments, based on market timing. Basically, we try to keep bucket one fully funded all of the time.

RBD: Was Vanguard your 401(k) provider and brokerage for all those years?

Fritz: Yes.

RBD: I recently transferred retirement assets from Vanguard for Fidelity to simplify my finances in light of completing our estate plan. I realized I had so many accounts and I had to consolidate.

Fritz: Absolutely simplify!

Another part of this, as we’re refilling bucket one, part of the way I do that is I simplify my portfolio. If I have one little mutual fund that has $10,000, I’ll just sell it.

Wherever there is additional complexity that’s not bringing additional value, use your withdrawal strategy to simplify your portfolio at the same time.

RBD: You’re somewhat famous for becoming a 401(k) millionaire. Do you consider that one of the keys to your early retirement? 

Fritz: To me it’s not the big a deal. You know, anybody with a good job and a 401(k) can be a 401(k) millionaire. If you start a job at 22, right out of college, and you’re in the 401(k) from the beginning and you’re contributing all along, it’s automatic. It’s not hard, it just takes time. You gotta be patient.

I think patience on the journey to FIRE is one of the hardest things. You get to the point where your investing is automated and growing. Then you’ve just gotta sit and wait.

Whether it’s in a 401(k) or other types of accounts.. nobody’s going to be a 55-year-old person in our (FIRE) community that’s not going to have a million dollars because we’re disciplined savers. It’s the power of compounding.

RBD: Do you worry for people who are in a hurry to declare early FIRE, nowadays, in this bull market? Especially people that have never been through a big market crash and recession that us older folks have experienced in the past?

Fritz: Yeah. And that’s why the emphasis on the safe withdrawal rate is important. Because if people think a 3.0% to 3.25% on a million bucks is enough… you’re gonna live on $30,000 for the rest of your life? Really? It’s not that much.

You really need like $2 to $3 million, and I’m not going to be a Suze Orman here, but you need enough in there so that you’re not unrealistic on your withdrawal rate expectations because you gotta know the markets are going to be cyclical and you gotta be able to endure a downturn or be willing to go back to work… and then hope you can get a decent job, because your skills are not going to be as marketable.

And by the way, if you want to go back and get another job, well guess what, nobody’s hiring after during the downturn or recession. 

RBD: What was the most challenging financial period during your retirement journey? Was there ever a point when you weren’t making enough money or experienced difficult financial periods? Being with a corporation for all that time, I’m guessing there weren’t many.

Fritz: No, surprisingly no. I think because we were so disciplined to never spend our entire income, even when I was making $21,000 out of college, I didn’t feel financially strapped because we were still spending less than we made.

At the time, I was living in a basement apartment, I had a roommate, and I drove a $500 Subaru. So I was always hyper-careful to not get into commitments that would take our entire paycheck. So we never felt financial strain. That’s why I had an emergency fund by the time I was age 23.

Then my Dad was a teacher, so I got the tuition exchange program which is free tuition to schools within the network and my parents paid for room and board. And that’s why we paid for our daughter because it’s time to pay it back.

RBD: Yeah, that’s the same way I feel. In fact, my parents paid for my college education with the expectation that I’d do the same for my kids.

Fritz: So I came out of college debt-free, and started making money within a month of graduating.

RBD: When I lost my job and spent four months as a full-time blogger, it felt very unstable to live without a regular paycheck. Then when I finally went back to work, there was this tranquility of having a salary again. It’s not total freedom, obviously, but it does allow you to live a comfortable life. How do you feel about a steady paycheck vs. becoming an entrepreneur?

Fritz: A steady paycheck gives you financial security and less stress about financials. But realistically, part of the price you pay is you’re putting up with a lot of corporate BS that you can potentially avoid as an entrepreneur.

So you’re not trouble-free. You’ve just got more troubles that are political rather than financial.

But also, the real security comes from always spending less than you earn. It doesn’t have to be a corporate job.

RBD: Have you figured out your retirement healthcare situation yet?

No, we’re still on COBRA but we’re getting close to the 18-month limit. We’re about to head out on a great American road trip and I don’t want to think about it! I’ll deal with it when I get back.

I had always built into my retirement cash flow the assumption that I’d have to pay for my own private health insurance.

We’re headed out to the Pacific Northwest to visit our daughter who had a baby recently. So we’ll be spending our summer out there. 

RBD: And you paid for your daughter’s college, right. Did you use 529 savings to pay for it? Were you able to cover everything with the 529 money?

I started saving in the 529 within about six months of her birth.

When she was in school, we wanted to drain down the 529 first. That covered the first two years. But at the same time, we were also saving money into a money market account, knowing that it would help pay for years three and four. So once the 529 was depleted, we already had the cash to pay for the final years. 

RBD: And how old were you when she graduated college? 

About 52. 

RBD: You recently completed U.S. train trip, you’re embarking on an epic road trip this summer, what’s next after the road trip? Any plans for international travel? What are some of your dream destinations?

I’ve got a million miles with Delta, so we can travel as much as we want. I did a lot of international travel throughout my career, so we were kind of burned out on it when I retired.

We plan a year at a time. We have ideas out there for 2020 and 2021, but we try to focus on the present. I wrote a post recently called Don’t Look Back (You’re Not Going that Way), and it’s all about not spending all your time looking back or looking into the future, but being in the present as much as you can. And retirement is the perfect time for that.

Because when you’re trying to get to retirement, you’re naturally thinking about the future. But once you’re in retirement, the future’s kind of bleak, right? We’re all gonna die! So enjoy the present because you’ve earned it.

That all said, we’re thinking about doing an Australia/New Zealand loop, or maybe going back to the U.K., or spend a month in Ireland and Scotland. Maybe Iceland.

RBD: So what are you up to now? What’s with this charity that your wife started?

My wife started a non-profit called Freedom for Fido. The charity builds fences for low-income families for free. It’s for people who have a dog but can’t afford a fence for it. Sometimes the dogs are living on a chain and not living a good life.

So we provide fencing and a dog house. We’ve been raising money and we just did our first build.

And one of my bucket items for retirement was to learn how to do video editing. We’ve done some traveling and I have a drone, so we have some cool videos, but I never knew how to do the video editing.

I bought some software and it’s been sitting on my computer for the past year. But now that this fence build was coming up, it was time to learn. So I made a 5-minute video of our first build.

I’m supporting my wife’s charity and learning a new skill. Never stop learning, right? There’s always stuff to learn, and retirement is a time when you can take your time to learn new things.

Here’s the video of our first build:

RBD: Thanks again for the conversation and for sharing your story about how to retire at age 55 with us.

You can learn more about Fritz’s retirement story at The Retirement Manifesto blog.

Photo via DepositPhotos used under license