how to deal with UN8 characters

the text edtior will tell you on which line it is
but is there a command to sort of cancel this line so that it is not seen

sometimes lines of doc might contains some internet address which may have some UNT-8 characters
which are good for documentation in the file

thanks

If you deal with file you wanna read with python

If the source file is UTF-8 encoded, open the file like:

f = open(filepath, encoding=“utf-8”)

This will prevent error messages like “can’t decode byte at …”

If the file uses BOM, use “utf-8-sig” instead!

If the script itself contains special chars

Load the py file into text editor, that should work

For other files, you usually need to add a signature like:

-- coding: utf-8 --

If a special char is in a commen, it shouldn’t matter at all.

If it is printed to console for instance however, it will give you “can’t decode character” error.
Add .encode() to it for a ascii-literals-only byte object.
You can also decode it again to string if necessary, and ignore errors:

print(“Some special chars:”, “你好”.encode().decode(“ascii”, errors=‘ignore’))

let me give you a situaton i get

many times i save some code snippet into a ODT file in libre office
find it easier to document it and add color ect,

so if i copy some text back to a py file it may contain some UNT-8 charaters some how
and it refuse to run and gives errors !

another is that if in some comments line with # or using “”"
then if i add an internet link it might also contains some UNT-8 characters again and
will give error!

in text editor it should give the line on which it is happening or not!

so you say that if i set at beginning of file

-- coding: utf-8 --

that it should be able to accept some UNT-8 characters!

i’ll test that

thanks

here is test and and still buggy !




 
import bpy

# -*- coding: utf-8 -*-  
 
 
print ()
print ('# -*- coding: utf-8 -*-  ')
print ()
print (' Test on UNT-8 characters')
 
 
 
 
"""

CooleyTukey_FFT_algorithm
http://en.wikipedia.org/wiki/Cooley


the last line with link has a Special character
which gives error !

thanks

I can’t see any special chars…

You shouldn’t copy RTF text, maybe paste into notepad, select all, copy, paste into blender?

most of the time they are not RTF file
more like DOC file

i tried with Notepad++ and i think there is a way to see where the unicode are located

and not certain why i get this error

is there a command to stop the error in blender text editor if there any unicode characters?

most of time this happen into the “”" “”" section not in the code part itself

try this file you might get the error
and cannot find where it is in text editor !

utf8test1.blend (531 KB)

thanks

it doesn’t matter what they are, but formatting usually carries over via clipboard if it’s MS Word or OO/LO Writer (rich text formatting).

The encoding info needs to be in the first line, latin charset works for me:

-- coding: 8859 --

Here’s a little script to remove some bad chars, should be fine as long as they are located in comments:

filepath = r"C:\Users\CoDEmanX\Desktop\utf-8test1.py"

file = open(filepath, "r")
out = open(filepath + ".cleared.py", "w")

text = file.read().encode("utf-8").decode("ascii", errors='ignore')
dic = text.maketrans('', '', '\x00\x01\x02\x03\x04\x05\x06\x07\x08\x0b\x0c\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xad')
out.write(text.translate(dic))

out.close()
file.close()

Text > Save as, save to file, use above script with adjusted path to that file and run to get a cleared version.