Milk helps coat the lining of your stomach, so it can help I would assume. But also don't drink too much water and eat smaller more frequent meals to help ease the symptoms. If it goes untreated, it can cause complications such as a risk of cancer due to stomach cell changes or gastrointestinal hemorrhage or ulcers. So I would definitely take it seriously. Eating more fruits and vegetables can help apparently, but nothing citrus-y, though as that can cause more acid indigestion.