Your syntax is wrong. You cant use return to return a value from a matlab function, you can only use it to exit exit the function therefore you need to define your output value before using the return statement.
ElbartoCode:function a = rec(n) if(n==1) a = 1; return else a = n * rec(n-1); end
Beat me by 2 minutes CB
I should also clarify that as in CB's code the return statement isn't necessary in this case, but I included it in my code to show how it should be used since I think it is where you are getting matlab confused with other languages like C or python etc.