United extended their lead at the top of the FA Women's National League Northern Premier Division to nine points thanks to early strikes from Beth Lumsden - her first for the club - and Paige Bailey-Gayle, whose goal was her fourth in three matches.
Midfielder Kelly was at the heart of everything United did going forward against Derby in a game which the Lasses dominated but struggled to put their opponents to bed.
"I think we had quite a few chances, especially second-half," Kelly admitted post-match. "We aren't going to reflect too badly on that.
"We got the result, we saw it out and three points is three points."
Kelly, a summer signing from Sunderland Women, added: "We just need to be a bit more clinical in games coming up.
"They're a really tough team to play against and were really organised. I think our game plan worked; we were patient and got the goals that we needed.
"It could've been more but we still got the three points."
Second-placed Nottingham Forest Women could only draw with Liverpool Feds on Sunday but Kelly is only focused on Newcastle taking care of their own business with seven league games left to play.
"(Forest's result) is extra motivation but we're trying to just focus on ourselves and not too much on what other teams are doing," she insisted.
"Obviously, it helps us out when teams are dropping points but it's nice to know we're getting the three points every week as that's all we can do."
SUPPORT NEWCASTLE WOMEN AT ST. JAMES' PARK
Newcastle United Women host Portsmouth Women at St. James' Park on Sunday, 25th February in the semi-finals of the FAWNL Cup.
Kick-off is at 2pm and you can get your tickets via book.nufc.co.uk.