Swansea striker Fernando Llorente could have joined Chelsea in January
Swansea striker Llorente only joined the club last summer but had the chance to jump ship midway through the season.
Chelsea were at risk of losing Diego Costa to China at the time and Conte was on the lookout for a quick fix.
But the Spaniard revealed to radio station Cadena SER that it was not the right time to move.
"We [Swansea] were in a delicate moment and it was logical that they would not let me leave," he said.
Chelsea line-up predicted to face Arsenal in the FA Cup final Thu, May 25, 2017
Click to reveal the predicted Chelsea line-up to face Arsenal in the FA Cup final
Play slideshow Getty Images 1 of 11
GK: Thibaut Courtois
I will never say no to a return to Spain
Llorente scored 15 goals for Swansea this season to help them avoid relegation – with four of those coming in their last five league games.
The former Bilbao and Sevilla star admits, however, that he would be temped by a return to La Liga.
"I have a year of contract [remaining] and I am very happy," he added.
"It is logical that offers could arrive but we will see.
Get Quotes on Home Insurance
"Let’s see what can happen in football. I hope things go well, if I stay or if I’m going.
"I will never say no to a return to Spain."
Fernando Llorente scored 15 goals for Swansea this season
Chelsea defender John Terry (left) will leave the club this summer
Swansea boss Paul Clement faces a tough summer ahead, with a number of his top stars potentially leaving the club.
Gylfi Sigurdsson is wanted at Everton – despite Swansea's keenness to keep the Icelander.
Clement has also said he is open to talks with John Terry over a possible move for the veteran Chelsea centre-back who is out of contract this summer.
"I will have a conversation with him. I've known him a long time," Clement said of Terry earlier this month.
And the manager is also keen to have Bafetimbi Gomis back at the club after his impressive loan spell at Marseille.