Different lumps can have different causes. If it is a boil, for example, you may need antibiotics, so do get it checked. In-growing hairs can develop after shaving. You may find it better to keep your armpit hair, or if you want to remove it, to dissolve it away using a depilatory cream. If you can stand the pain, waxing to strip the hairs out will also tend to encourage normal regrowth of hairs as they grow through from the root, rather than from a partially developed hair shaft.