Context information is significant for semantic extraction and recovery of messages in semantic communication. However, context information is not fully utilized in the existing semantic communication systems since relationships between sentences are often ignored. In this paper, we propose an Extended Context-based Semantic Communication (ECSC) system for text transmission, in which context information within and between sentences is explored for semantic representation and recovery. At the encoder, self-attention and segment-level relative attention are used to extract context information within and between sentences, respectively. In addition, a gate mechanism is adopted at the encoder to incorporate the context information from different ranges. At the decoder, Transformer-XL is introduced to obtain more semantic information from the historical communication processes for semantic recovery. Simulation results show the effectiveness of our proposed model in improving the semantic accuracy between transmitted and recovered messages under various channel conditions.