I have never been one of those dudes that loves aviation for the fact that it is aviation. However, flying an airplane with "US Air Force" painted on the side is the coolest thing ever.
Where you said you enjoy challenges, that makes me think that you would love to be a pilot. You'll go through UPT and think that starting the airplane is going to make you bleed from the ears at first. Before long you'll be doing aerobatics and flying in the weather off of some goofy looking drawing on a piece of paper they keep calling a "plate". And right about the time you get the hang of that, they'll add another airplane into the mix. Once you get the hang of that, you'll move onto a faster jet (usually) and it starts all over. Then when you're mister shit hot pilot dude, you'll realize that all that stuff you just learned for a year is just how to get to/from the actual work in the airplane and you really don't know a damn thing.
Repeat the same process through IFF/RTU/2 FLUG/ ACUG/ IPUG/Airdrop/ whatever. Very rewarding and constantly challenging.