Train until your confident with your equipment. If you don't feel comfortable with a loaded 1911 on your hip, then don't carry it or don't carry it loaded. It's about what your can trust yourself with.
I carry my 1911 tac ops everyday loaded, safety on and have zero worries of it failing me, or me failing it.
Again, it's just about your comfort level and trust In yourself.
You won't have time to rack that slide if it came down to it. You need about 25' just to draw and fire your gun from an attacker charging your with a knife. There has been lots of studies done on this, and you can find the old FBI data on YouTube of you dig deep enough. Now, imagine someone with a gun attacking you. Still think you will have enough time to rack the slide and defend yourself?
I highly recommend you get a gun your comfortable with to carry loaded.