Stress, worries and problems in the partnership can decrease sexual desire both in men and women or even cause it to disappear completely. But the libido can also be adversely impacted by physical and psychological diseases such as metabolic diseases, lack of hormones or depression.