Taiwan elects first female president
Ms Tsai, 59, leads the Democratic Progressive Party (DPP) that wants independence from China.
In her victory speech, she vowed to preserve the status quo in relations with China, adding Beijing must respect Taiwan's democracy and both sides must ensure there are no provocations.
China sees the island as a breakaway province - which it has threatened to take back by force if necessary.
In her speech, Ms Tsai hailed a "new era" in Taiwan and pledged to co-operate with other political parties on major issues.
The will of the Taiwanese people would be the basis for relations with China, Ms Tsai said.
"I also want to emphasise that both sides of the Taiwanese Strait have a responsibility to find mutually acceptable means of interaction that are based on dignity and reciprocity.
"We must ensure that no provocations or accidents take place," Ms Tsai said, warning that "any forms of suppression will harm the stability of cross-strait relations".
She thanked the US and Japan for their support and vowed Taiwan would contribute to peace and stability in the region.
Ms Tsai had a commanding lead in the vote count when Eric Chu of the ruling Kuomintang (KMT) admitted defeat.
Mr Chu congratulated Tsai Ing-wen and announced he was quitting as KMT head. Taiwan's Premier Mao Chi-kuo also resigned.
The election came just months after a historic meeting between the leaders of Taiwan and China.
However, the flagging economy as well as Taiwan's relationship with China both played a role in the voters' choice, correspondents say.
The KMT has been in power for most of the past 70 years and has overseen improved relations with Beijing - Ms Tsai's is only the second-ever victory for the DPP.
The first was by pro-independence advocate Chen Shui-bian; during his time as president between 2000 and 2008 tensions with China escalated.
The victory by Tsai Ing-wen marks a defeat for not only the pro-unification ruling party KMT but also China.
Despite the past eight years of reduced tensions and much improved relations built by the KMT and China, Taiwanese voters have voted for Ms Tsai from the pro-independence party instead. Basically, they've voted to keep Beijing at a distance.
This reflects not only widespread dissatisfaction with President Ma Ying-jeou of the KMT over insufficient measures to improve the lacklustre economy, low wages and widening wealth gap - it also reflects growing worries by Taiwanese people that the island may become too economically dependent on China and that this will make it hard for Taiwan to fend off pressures by Beijing to reunify with it one day.
The message voters have sent Beijing is that, while they want reduced tensions and good relations, they cherish Taiwan's sovereignty, democracy and self-rule even more.
The challenge now is for Ms Tsai to find a way to work with China, the island's biggest export market, trade partner and security threat.
Ms Tsai, a former scholar, has said she wants to "maintain [the] status quo" with China.
She became chairwoman of the DPP in 2008, after it saw a string of corruption scandals.
She lost a presidential bid in 2012 but has subsequently led the party to regional election victories. She has won increased support from the public partly because of widespread dissatisfaction over the KMT and President Ma Ying-jeou's handling of the economy and widening wealth gap.
Saturday's polls come after a historic meeting between President Ma and Chinese President Xi Jinping in Singapore in November for talks that were seen as largely symbolic - the first in more than 60 years.
Eric Chu, 54, is the mayor of New Taipei City and stepped up to become chairman of the party in October.
The KMT has lost its majority in the legislature for the first time in history, BBC reported.
The former accounting professor was seen as popular with young people in the party, but had been unable to change public opinion that is increasingly unhappy with the party's friendly stance towards China and the island's economic travails.
In 2014, hundreds of students occupied the parliament in the largest show of anti-Chinese sentiment on the island for years. Labelled the Sunflower Movement, protesters demanded more transparency in trade pacts negotiated with China.
Taiwan for all practical purposes been independent since the end of the Chinese civil war in 1949, when the defeated Nationalist government fled to the island as the Communists, under Mao Zedong, swept to power.