Although it's inevitable that your testosterone levels will decline as you age, you can stave off some of that decline by staying healthy. A 2007 study in the Journal of Clinical Endocrinology and Metabolism found that a man's testosterone levels were significantly affected by obesity, as well as diabetes and other chronic illnesses.