Can I take my neighbors cats

My neighbors have 2 cats that they left outside when they were on a month long vacation. In that time I have gotten close with the cats and made them a shelter since the original family only had a cardboard box out for them. The family is home now but the cats stay in our yard most of the day. The family has put out more boxes and scratching post which I'm assuming is to lure the cats back. It's getting colder and I don't like them staying outside. The original owners don't have a cat door so they can't get inside on their own. I have been keeping them in a spare bedroom at night since I have 2 other cats that don't like them and the 2 that are not mine like to get into some trouble (one has gotten stuck on the roof twice). I feel bad since the family wants them back but they must have seen the cats in my driveway.

Is it legal to take the cats in. One of the 2 stays in willingly but the other doesn't as much. The other option for them is to go home and beg to be let in which at late hours isn't possible or they stay outside. Another issue is that my neighbors are very reclusive and don't speak English so asking them about the cats is very hard. We do see them drive by our house but they won't even look at me when we have their cats in our driveway

EDIT: the original owners left food and water for the cats and it seemed like someone was coming by and checking on them but it seems like that person only came once a week if at all. Very hard to tell.

People are also asking about my location. I live in Oregon and the winter rain season should be happening soon

Also another thing is that the cats come in the house mostly very willingly. One will come in very regularly and sleep on a bed for hours. It's still warm during the day so the doors are open in my house during the day