Not at all. You can study abroad if you only know English, and you won’t be limited to English-speaking countries like England or Australia, either. There are study abroad programs available in English all over the world. English-taught courses will be advertised on the university’s website and can sometimes be searched for using a centralized database run by a national agency. If you do end up in a non-English speaking county, you will most likely pick up parts of the language (another advantage!).