When you have done something for a long time it is rather easy to keep doing it. However, when we you get out of doing that particular habit it is just as easy to discontinue it altogether.
For instance, I ran cross country and track for three years in high school but once I graduated and got out of the running routine it became much more difficult to motivate myself. As a result I am currently out of shape and don't really desire to go on runs in the morning.
This same principle applies to church attendance. Just as I love to run, I love going to church. It keeps me in shape spiritually. On my mission I have seen many who have left the path of church attendance. I have also seen many choose to come back. At first it is often hard to return but as they have worked through that initial discomfort they've been able to reap the benefits of going to church.
Even though it might be hard at first the blessings of going to church are well worth it in the end.