Moon quickly reacted to Kim's remarks, saying it was "a response to our proposal to turn the Pyeongchang Olympic Games into an epoch-making opportunity to improve inter-Korean relations and establish peace".
South Korea and the DPRK fielded a joint women's ice hockey team, the first Olympic team between the two Koreas.
"Before posing together for the photo, President Moon explained the meaning of the art work to Kim Yo Jong and Kim Yong Nam", said a presidential aide.
Otto Warmbier's father has spoken about North Korea's participation in the Winter Olympics and said his presence at the Games is not "political". That led to days of negotiations between the two countries, which in turn secured the north's participation at the 2018 Winter Olympics.
The Winter Olympics opened earlier today (local time) with cultural and artistic performances as well as the customary parade of athletes, which was the delegations from the Democratic People's Republic of Korea and the Republic of Korea under one flag, carried together by a sportsperson from each team.
If that happens, then the United States might lose its seat in any future talks to end North Korea's nuclear program - and it might harm relations with a staunch ally in the process.
Pence is viewing the short-track competition, in which eight Americans are participating. "I think Kim Jong-un has betrayed South Korea so many times and we can not guarantee what will happen after the Games".
"I think this overture towards South Korea is partly sanctions-related, and also related to the fact that it's clear a divergence has developed between Washington and Seoul's most keenly desired goals in the near term", said Andray Abrahamian, a research fellow at Pacific Forum CSIS in Hawaii. His sister verbally delivered his offer at the meeting. But North Korea's record on human rights and the growing threat from its nuclear weapons program appeared out of mind.
Moon and Pence didn't discuss an invitation extended by Kim Jong Un to meet with the South Korean leader in Pyongyang, said a senior White House official who requested anonymity to discuss internal deliberations.
Though it may take months to figure out who is responsible for the attacks and why they were executed, experts say the evidence points to the "hallmarks of a nation state", The New York Times reported.
The discussions from Saturday's meeting weren't immediately released.
Pence spent the days leading up to the Pyeongchang Olympics warning that the North was trying to "hijack the message and imagery" of the event with its "propaganda".
The officials toasted with the Korean liquor "soju" and were served buckwheat and potato pancakes, dried pollock soup, grilled halibut and other dishes.
This item corrects that Kim Yo Jong is the leader's sister.
The meeting followed a brief encounter between the two parties at the Olympic Opening Ceremony on Friday, in which Moon twice shook hands with Yo-jong.
The luncheon also includes Choe Hwi, chairman of the North's National Sports Guidance Committee, and Ri Son Gwon, chairman of the agency that deals with inter-Korean affairs.