A too much keyword stuffed page is linking towards my site. Will it be a problem for my site?
If its providing a do-follow link then disavow it as soon as possible.
If the hosting pages are providing no value except from providing you with a link. Then just disavow them.
Google is aware of the keyword stuffed page that is linking towards your site and will ignore its low quality links.